[elixir-core:8317] [Proposal] Allow guards to be interspersed anywhere in clause heads

[Christopher Keele]

This is a more formal write-up of the discussion started with José here 
. Interested in all 
feedback, potential use-cases, and syntactic edge-cases!


Proposal

Allow `*when conditions...*` to be placed anywhere within the patterns used 
in functions, cases, and other such match head constructs—in addition to 
the suffix-only version we support today.

I believe this can be done by the compiler today as a backwards-compatible 
enhancement with no change to the parser.

Synopsis

I'm proposing we allow guard clauses anywhere within match patterns, as 
well as following them, so that the AST for match heads containing guards 
can be composed easily.

To show rather than tell, assuming you have AST for the following:

X = x when x > 0
Y = y when y > 1

Both *X* and *Y* are valid, stand-alone match heads. However, they are not 
also valid stand-alone match patterns: there is no easy way to combine them 
to form a new 2-arity match. The naive approach would be:

XY = (x when x > 0, y when y > 1)

While this is not currently a valid match head, once can hoist any internal 
guards into a valid form like:

XY2 = (x, y) when x > 0 and y > 1

This proposal explores allowing *XY* as valid Elixir code and rewriting it 
to *XY2* at compile time.


Terminology

To talk about this precisely I'm going to appropriate some ETS terminology 
to reference Elixir syntax constructs I don't have canonical names for. Let 
me know if you are aware of the correct terms for these!

- a *match specification* is defined as a *match head -> body* pair
- a *match head* is defined as a *match pattern* optionally followed by *guard 
clauses*
- a *match pattern* is defined as a comma delimited series of expressions 
allowed in match contexts
- a *guard clause* is defined as a *when conditions* expression formulated 
of functions allowed in guard contexts

Technically, what I'm proposing is to loosen the restriction that guard 
clauses must follow match patterns, by allowing match heads themselves to 
recursively be valid expressions in parameters lists. The compiler can 
extract all guard clauses found within a match head, leaving only a valid 
match pattern, and combine the extracted guards with any existing ones to 
create a new valid match head with equivalent semantics.


Parsing

Currently, both of these (and all other variations I can think of) are 
already valid syntax to the parser:

fn x when x > 0, y when y > 1 -> #... end
def name(x when x > 0, y when y > 1), do: #...

Only the compiler keeps you from running a program with these internal 
guards; it can be parsed into AST with the exact precedence we want without 
a hitch.


Grammar

The only ambiguity I can think of would be the following:

fn x when x > 0, y when y > 1 when y < 3 -> #... end

It is not clear where whether the intent was to create an 'or' multi-guard 
around the entire parameters list, or to guard both the last parameter and 
the parameters at large with two separate clauses. The parenthesis 
conventionally always used in *def*s resolves the ambiguity. I am open to 
ideas on how to handle this situation, though personally I envision a 
compile-time warning and treating it as a multi-guard as this is most 
consistent with the precedence of *when*. I don't see it coming up too 
often in generated code.

It is worth observing that while the following technically has the same 
ambiguity:

fn x when x > 0, y when y > 1 -> #... end

however you decide to treat the guard after the second parameter, the 
resulting guards post-rewrite will be semantically equivalent.


Rewriting

Since everything up to this point is already valid, I suspect the rewrite 
could be done in a single place 

 
in the compiler with no further changes to any other code.

The algorithm I have in mind is to simply walk the AST outside-in, removing 
each set of consecutive guards it finds, and exploding the permutation of 
each sets' multi-guards out to create new trailing clauses, then *and*ing all 
terms together in each new guard clause. In the most common case that 
simply entails prefacing the trailing guards with any interior guard, all 
*and*ed together.

This approach is intentionally naive about what variables are referenced in 
which guards where. Anything that produces a valid guard in the end can 
fly, even if generated code produces them in odd or unexpected places 
within the params list.

Simple extraction:

def name(%{foo: bar when is_integer(bar), fizz: buzz when is_integer(buzz)})
  when bar + buzz > 100

def name(%{foo: bar, fizz: buzz})
  when (is_integer(bar) and is_integer(buzz)) and (bar + buzz > 100)


In order to handle mutli-guards correctly we'd have to get a little *n^m *but 
I doubt even the most ambitious metaprogramming would ever need to generate 
code like that.

Matrix of multi-guards with 

Re: [elixir-core:8317] [Proposal] Allow guards to be interspersed anywhere in clause heads

[Allen Madsen]

I believe `when` with `=` is visually ambiguous and works different from
how people would expect it to.

The way I read the result of this AST:

def foo(%{x: x} when x > 1 = result), do: 1

{:def, [context: Elixir, import: Kernel],
 [
   {:foo, [context: Elixir],
[
  {:when, [],
   [
 {:%{}, [], [x: {:x, [], Elixir}]},
 {:=, [],
  [
{:>, [context: Elixir, import: Kernel], [{:x, [], Elixir}, 1]},
{:result, [], Elixir}
  ]}
   ]}
]},
   [do: 1]
 ]}

It means:

def foo(%{x: x} when (x > 1 = result)), do: 1

Instead of:

def foo((%{x: x} when x > 1) = result), do: 1

Allen Madsen
http://www.allenmadsen.com


On Mon, Sep 24, 2018 at 10:13 AM Christopher Keele 
wrote:

> This is a more formal write-up of the discussion started with José here
> . Interested in all
> feedback, potential use-cases, and syntactic edge-cases!
>
>
> Proposal
>
> Allow `*when conditions...*` to be placed anywhere within the patterns
> used in functions, cases, and other such match head constructs—in addition
> to the suffix-only version we support today.
>
> I believe this can be done by the compiler today as a backwards-compatible
> enhancement with no change to the parser.
>
> Synopsis
>
> I'm proposing we allow guard clauses anywhere within match patterns, as
> well as following them, so that the AST for match heads containing guards
> can be composed easily.
>
> To show rather than tell, assuming you have AST for the following:
>
> X = x when x > 0
> Y = y when y > 1
>
> Both *X* and *Y* are valid, stand-alone match heads. However, they are
> not also valid stand-alone match patterns: there is no easy way to combine
> them to form a new 2-arity match. The naive approach would be:
>
> XY = (x when x > 0, y when y > 1)
>
> While this is not currently a valid match head, once can hoist any
> internal guards into a valid form like:
>
> XY2 = (x, y) when x > 0 and y > 1
>
> This proposal explores allowing *XY* as valid Elixir code and rewriting
> it to *XY2* at compile time.
>
>
> Terminology
>
> To talk about this precisely I'm going to appropriate some ETS terminology
> to reference Elixir syntax constructs I don't have canonical names for. Let
> me know if you are aware of the correct terms for these!
>
> - a *match specification* is defined as a *match head -> body* pair
> - a *match head* is defined as a *match pattern* optionally followed by *guard
> clauses*
> - a *match pattern* is defined as a comma delimited series of expressions
> allowed in match contexts
> - a *guard clause* is defined as a *when conditions* expression
> formulated of functions allowed in guard contexts
>
> Technically, what I'm proposing is to loosen the restriction that guard
> clauses must follow match patterns, by allowing match heads themselves to
> recursively be valid expressions in parameters lists. The compiler can
> extract all guard clauses found within a match head, leaving only a valid
> match pattern, and combine the extracted guards with any existing ones to
> create a new valid match head with equivalent semantics.
>
>
> Parsing
>
> Currently, both of these (and all other variations I can think of) are
> already valid syntax to the parser:
>
> fn x when x > 0, y when y > 1 -> #... end
> def name(x when x > 0, y when y > 1), do: #...
>
> Only the compiler keeps you from running a program with these internal
> guards; it can be parsed into AST with the exact precedence we want without
> a hitch.
>
>
> Grammar
>
> The only ambiguity I can think of would be the following:
>
> fn x when x > 0, y when y > 1 when y < 3 -> #... end
>
> It is not clear where whether the intent was to create an 'or' multi-guard
> around the entire parameters list, or to guard both the last parameter and
> the parameters at large with two separate clauses. The parenthesis
> conventionally always used in *def*s resolves the ambiguity. I am open to
> ideas on how to handle this situation, though personally I envision a
> compile-time warning and treating it as a multi-guard as this is most
> consistent with the precedence of *when*. I don't see it coming up too
> often in generated code.
>
> It is worth observing that while the following technically has the same
> ambiguity:
>
> fn x when x > 0, y when y > 1 -> #... end
>
> however you decide to treat the guard after the second parameter, the
> resulting guards post-rewrite will be semantically equivalent.
>
>
> Rewriting
>
> Since everything up to this point is already valid, I suspect the rewrite
> could be done in a single place
> 
> in the compiler with no further changes to any other code.
>
> The algorithm I have in mind is to simply walk the AST outside-in,
> removing each set of consecutive guards it finds, and exploding the
> permutation of each sets' multi-guards ou

Re: [elixir-core:8319] [Proposal] Allow guards to be interspersed anywhere in clause heads

[Christopher Keele]

That's a good point. I suspect the convention around manual usage would 
converge around:

*def foo(%{x: x} = result when x > 1), do: 1*

which parses as expected. This version feels more natural to me.

I don't think the other form would accidentally be introduced while 
injecting one snippet of AST, because the tree would never be constructed 
the wrong way. String-based expression interpolation might make that 
mistake, but if you're metaprogramming with literal strings of code you've 
already made a mistake.

I hadn't typed out an assignment case with inline guards, it does feel like 
it's doing too much in a short space with few grammar tokens to help 
understand it. That's a good argument for intentionally discouraging these 
constructs outside of metaprogramming, rather than adopting it as a common 
usage language feature.

-- 
You received this message because you are subscribed to the Google Groups 
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elixir-lang-core/f2fafacb-3302-4c03-9b1b-ed766a896cc2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [elixir-core:8319] [Proposal] Allow guards to be interspersed anywhere in clause heads

[Allen Madsen]

If the goal were primarily macro usage, wouldn't this type of thing already
be possible with a macro today?

Allen Madsen
http://www.allenmadsen.com


On Mon, Sep 24, 2018 at 11:34 AM Christopher Keele 
wrote:

> That's a good point. I suspect the convention around manual usage would
> converge around:
>
> *def foo(%{x: x} = result when x > 1), do: 1*
>
> which parses as expected. This version feels more natural to me.
>
> I don't think the other form would accidentally be introduced while
> injecting one snippet of AST, because the tree would never be constructed
> the wrong way. String-based expression interpolation might make that
> mistake, but if you're metaprogramming with literal strings of code you've
> already made a mistake.
>
> I hadn't typed out an assignment case with inline guards, it does feel
> like it's doing too much in a short space with few grammar tokens to help
> understand it. That's a good argument for intentionally discouraging these
> constructs outside of metaprogramming, rather than adopting it as a common
> usage language feature.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elixir-lang-core+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/f2fafacb-3302-4c03-9b1b-ed766a896cc2%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elixir-lang-core/CAK-y3CsZJV4WXp5mtSqMtxZ_%3DXHoytyGmKET8pDXWQutUax9Ng%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [elixir-core:8321] [Proposal] Allow guards to be interspersed anywhere in clause heads

[Fernando Tapia Rico]

IIRC, macros do not allow to abstract both the pattern match and the guard 
within the same expression. For example, the Elixir codebase contains a 
helper function that abstracts the pattern match, the guard and the "read" 
function for dates and times separately. Here is the helper function 
https://github.com/elixir-lang/elixir/blob/master/lib/elixir/lib/calendar/iso.ex#L39-L70;
 
and here is an example of its 
usage 
https://github.com/elixir-lang/elixir/blob/master/lib/elixir/lib/calendar/datetime.ex#L522-L531.

Besides, as described by José Valim in here 
,
 
internal guards would allow to write macros like:

def my_fun(my_macro())

which would be expanded to:

def my_fun(my_var when my_var in [:foo, :bar])

and rewritten by the Elixir compiler to:

def my_fun(my_var) when my_var in [:foo, :bar]

However, I'm not sure how common are those use cases 🤔.

On Monday, September 24, 2018 at 8:22:07 PM UTC+2, Allen Madsen wrote:
>
> If the goal were primarily macro usage, wouldn't this type of thing 
> already be possible with a macro today?
>
> Allen Madsen
> http://www.allenmadsen.com
>
>
> On Mon, Sep 24, 2018 at 11:34 AM Christopher Keele  > wrote:
>
>> That's a good point. I suspect the convention around manual usage would 
>> converge around:
>>
>> *def foo(%{x: x} = result when x > 1), do: 1*
>>
>> which parses as expected. This version feels more natural to me.
>>
>> I don't think the other form would accidentally be introduced while 
>> injecting one snippet of AST, because the tree would never be constructed 
>> the wrong way. String-based expression interpolation might make that 
>> mistake, but if you're metaprogramming with literal strings of code you've 
>> already made a mistake.
>>
>> I hadn't typed out an assignment case with inline guards, it does feel 
>> like it's doing too much in a short space with few grammar tokens to help 
>> understand it. That's a good argument for intentionally discouraging these 
>> constructs outside of metaprogramming, rather than adopting it as a common 
>> usage language feature.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elixir-lang-core" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elixir-lang-co...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elixir-lang-core/f2fafacb-3302-4c03-9b1b-ed766a896cc2%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elixir-lang-core/7fda093f-03b5-4d1d-9c16-73c9475ca6a7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [elixir-core:8321] [Proposal] Allow guards to be interspersed anywhere in clause heads

[Louis Pilfold]

Hey

Macros are more than capable of providing this functionality, you'd just
need to define your own `def` like macro that generates the desired
function.

```
MyLib.def my_fun(my_var when my_var in [:foo, :bar])
```

Cheers,
Louis

On Mon, 24 Sep 2018 at 20:01 Fernando Tapia Rico 
wrote:

> IIRC, macros do not allow to abstract both the pattern match and the guard
> within the same expression. For example, the Elixir codebase contains a
> helper function that abstracts the pattern match, the guard and the "read"
> function for dates and times separately. Here is the helper function
> https://github.com/elixir-lang/elixir/blob/master/lib/elixir/lib/calendar/iso.ex#L39-L70;
> and here is an example of its usage
> https://github.com/elixir-lang/elixir/blob/master/lib/elixir/lib/calendar/datetime.ex#L522-L531
> .
>
> Besides, as described by José Valim in here
> ,
> internal guards would allow to write macros like:
>
> def my_fun(my_macro())
>
> which would be expanded to:
>
> def my_fun(my_var when my_var in [:foo, :bar])
>
> and rewritten by the Elixir compiler to:
>
> def my_fun(my_var) when my_var in [:foo, :bar]
>
> However, I'm not sure how common are those use cases 🤔.
>
> On Monday, September 24, 2018 at 8:22:07 PM UTC+2, Allen Madsen wrote:
>
>> If the goal were primarily macro usage, wouldn't this type of thing
>> already be possible with a macro today?
>>
>> Allen Madsen
>> http://www.allenmadsen.com
>>
>>
>> On Mon, Sep 24, 2018 at 11:34 AM Christopher Keele 
>> wrote:
>>
> That's a good point. I suspect the convention around manual usage would
>>> converge around:
>>>
>>> *def foo(%{x: x} = result when x > 1), do: 1*
>>>
>>> which parses as expected. This version feels more natural to me.
>>>
>>> I don't think the other form would accidentally be introduced while
>>> injecting one snippet of AST, because the tree would never be constructed
>>> the wrong way. String-based expression interpolation might make that
>>> mistake, but if you're metaprogramming with literal strings of code you've
>>> already made a mistake.
>>>
>>> I hadn't typed out an assignment case with inline guards, it does feel
>>> like it's doing too much in a short space with few grammar tokens to help
>>> understand it. That's a good argument for intentionally discouraging these
>>> constructs outside of metaprogramming, rather than adopting it as a common
>>> usage language feature.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elixir-lang-core" group.
>>>
>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to elixir-lang-co...@googlegroups.com.
>>
>>
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elixir-lang-core/f2fafacb-3302-4c03-9b1b-ed766a896cc2%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elixir-lang-core+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/7fda093f-03b5-4d1d-9c16-73c9475ca6a7%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFA4Eh_zuzbR2ykUzC7UXC%3DcK7NmuyLrXjH%3DgkFe8%2BJzYg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [elixir-core:8323] [Proposal] Allow guards to be interspersed anywhere in clause heads

[Christopher Keele]

 > *If the goal were primarily macro usage, wouldn't this type of thing 
already be possible with a macro today?*

> *Macros are more than capable of providing this functionality, you'd just 
need to define your own `def` like macro that generates the desired 
function.*

This is absolutely true, and expat does something similar today. It could 
be done easier and better in core, of course. I think the heart of the 
proposal is: given that this is straightforwardly possible today, and the 
implementation identical and completely deterministic no matter the 
application, is there a good reason *not* to expand the expressivity of 
guard clauses in the core of the language beyond that which erlang offers?

The macro-authors-over-common-coding argument is me simply saying, let's 
consider this from the perspective of increasing the language's 
extensibility first, since I suspect the discussion around whether or not 
it makes day-to-day coding more productive to be far more opinion-oriented.

I am not sold on the idea myself, which is why I dug into the 
implementation a bit so I could present it better. It absolutely could be 
done via macro, but I think it exists at an interesting low-hanging fruit, 
low-risk way to make the language itself more powerful today.

> *IIRC, macros do not allow to abstract both the pattern match and the 
guard within the same expression.*

You do a better job of summarizing how this could be used in core today 
than I did. I am not sure how common the use-case is either, but I do have 
one concrete example in expat and I wonder what might embraced in the 
future if this purely historical grammatical restriction were lifted.

-- 
You received this message because you are subscribed to the Google Groups 
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elixir-lang-core/9e9be43d-b772-480c-afe9-bb07a7e01119%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [elixir-core:8323] [Proposal] Allow guards to be interspersed anywhere in clause heads

[Louis Pilfold]

Hey

> is there a good reason *not* to expand the expressivity of guard clauses
in the core of the language beyond that which erlang offers?

I've no strong opinion on the proposal, but I'd just like to point out
that expressivity
has a formal CS definition and implementing this proposal will not increase
the expressive power of guard clauses as they will not be capable of any
functionality they did not have before.

This is a matter of style and syntax.

Cheers,
Louis

On Mon, 24 Sep 2018 at 21:13 Christopher Keele 
wrote:

>  > *If the goal were primarily macro usage, wouldn't this type of thing
> already be possible with a macro today?*
>
> > *Macros are more than capable of providing this functionality, you'd
> just need to define your own `def` like macro that generates the desired
> function.*
>
> This is absolutely true, and expat does something similar today. It could
> be done easier and better in core, of course. I think the heart of the
> proposal is: given that this is straightforwardly possible today, and the
> implementation identical and completely deterministic no matter the
> application, is there a good reason *not* to expand the expressivity of
> guard clauses in the core of the language beyond that which erlang offers?
>
> The macro-authors-over-common-coding argument is me simply saying, let's
> consider this from the perspective of increasing the language's
> extensibility first, since I suspect the discussion around whether or not
> it makes day-to-day coding more productive to be far more opinion-oriented.
>
> I am not sold on the idea myself, which is why I dug into the
> implementation a bit so I could present it better. It absolutely could be
> done via macro, but I think it exists at an interesting low-hanging fruit,
> low-risk way to make the language itself more powerful today.
>
> > *IIRC, macros do not allow to abstract both the pattern match and the
> guard within the same expression.*
>
> You do a better job of summarizing how this could be used in core today
> than I did. I am not sure how common the use-case is either, but I do have
> one concrete example in expat and I wonder what might embraced in the
> future if this purely historical grammatical restriction were lifted.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elixir-lang-core+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/9e9be43d-b772-480c-afe9-bb07a7e01119%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFDw%2B30jQ%2BhfsLS7OxXSFf3gT7XSxBZr-qGuBC6_kY-Tww%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.