[elixir-core:7910] Enhancement request: parse character offsets for AST node

2018-02-26 Thread Serge Smetana
Hi,

I would like Elixir AST nodes to include begin and end parse character 
offsets.
This would make it easier to write refactoring tools that modify parts of
Elixir source files.

Usecase:

We have a library https://github.com/assert-value/assert_value_elixir that 
is
able to create and update expected values in tests. For example, given the
following:

assert_value 2 + 2 == 2 + 3

our code will find the location of `2 + 3` and replace it with `4`:

assert_value 2 + 2 == 4

Today determining the location of `2 + 3` in the source file is difficult.  
For
now, we use a custom-made parser which processes code char by char until it 
gets
value matching AST:
https://github.com/assert-value/assert_value_elixir/blob/master/lib/assert_value/parser.ex
But if we had the parse offsets it would be much easier.

Proposed interface:

Probably need two sets of offsets, exclusive and inclusive of children.  For
each probably best to store beginning offset and length. Will need 
reasonable
handling for parentheses and other tokens that do not make it into AST.

Existing implementation:

Elixir AST nodes do have useful info on this already. We use the "line:" 
which
is very helpful. We don't use "column:", it did not seem useful given our
implementation.  We may be missing something obvious here.

Details:

In Elixir 1.6 compiled code AST has only function line number in meta. Even
"columns: true" in Code.string_to_quoted gives only function starting column
without information about arguments.

Consider the following code:

Code.string_to_quoted!("(41.00 == 42.)", columns: true)
#=> {:==, [line: 1, column: 8], [41.0, 42.0]}

>From the AST you don't know where to find 41.00 and 42. in a code.
Column information does not help. AST values of 41.0 and 42.0 don't have
information about how original values were formatted.

Thanks,
Serge

-- 
You received this message because you are subscribed to the Google Groups 
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elixir-lang-core/4dbc37cc-6630-4989-acc0-c4acfd719a5b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [elixir-core:7910] Enhancement request: parse character offsets for AST node

2018-02-26 Thread Louis Pilfold
Hi there

Given we have the formatter now I would be tempted to avoid editing strings
and instead update the AST of the file, render that to a string, format it
and write it back to the file. Would be more reliable.

Cheers,
Louis

On Mon, 26 Feb 2018, 12:11 Serge Smetana,  wrote:

> Hi,
>
> I would like Elixir AST nodes to include begin and end parse character
> offsets.
> This would make it easier to write refactoring tools that modify parts of
> Elixir source files.
>
> Usecase:
>
> We have a library https://github.com/assert-value/assert_value_elixir
> that is
> able to create and update expected values in tests. For example, given the
> following:
>
> assert_value 2 + 2 == 2 + 3
>
> our code will find the location of `2 + 3` and replace it with `4`:
>
> assert_value 2 + 2 == 4
>
> Today determining the location of `2 + 3` in the source file is
> difficult.  For
> now, we use a custom-made parser which processes code char by char until
> it gets
> value matching AST:
>
> https://github.com/assert-value/assert_value_elixir/blob/master/lib/assert_value/parser.ex
> But if we had the parse offsets it would be much easier.
>
> Proposed interface:
>
> Probably need two sets of offsets, exclusive and inclusive of children.
> For
> each probably best to store beginning offset and length. Will need
> reasonable
> handling for parentheses and other tokens that do not make it into AST.
>
> Existing implementation:
>
> Elixir AST nodes do have useful info on this already. We use the "line:"
> which
> is very helpful. We don't use "column:", it did not seem useful given our
> implementation.  We may be missing something obvious here.
>
> Details:
>
> In Elixir 1.6 compiled code AST has only function line number in meta. Even
> "columns: true" in Code.string_to_quoted gives only function starting
> column
> without information about arguments.
>
> Consider the following code:
>
> Code.string_to_quoted!("(41.00 == 42.)", columns: true)
> #=> {:==, [line: 1, column: 8], [41.0, 42.0]}
>
> From the AST you don't know where to find 41.00 and 42. in a code.
> Column information does not help. AST values of 41.0 and 42.0 don't have
> information about how original values were formatted.
>
> Thanks,
> Serge
>
> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elixir-lang-core+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/4dbc37cc-6630-4989-acc0-c4acfd719a5b%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFDcupN2SX%2Bivg%2BTJ7Q0LGbxARjAgRz-uta2ck8L%3DaTmKg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.