Re: [Haskell-cafe] [Haskell] ANNOUNCE: haskell-src-exts 1.14.0
On Aug 20, 2013, at 02:19 , Niklas Broberg wrote:
> Sadly not - it's theoretically impossible. The fact that you can put comments
> literally wherever, means that it's impossible to treat them as nodes of the
> AST. E.g.
>
> f {- WHERE -} x = -- WOULD
> -- THESE
> do -- COMMENTS
> a {- END -} <- g x -- UP
> return {- ? -} a
"Theoretically impossible". I wouldn't say so. In fact, a system like this
was implemented for BETA in the mid 1980'es [1]. The comments where
attached to the nearest AST node in an expanded AST. While not _perfect_,
it worked pretty well. Granted, it likely is much harder to do for Haskell than
BETA, but impossible is a strong word.
Tommy
[1] http://www.cs.au.dk/~beta/doc/mjolner-overview/mjolner-overview.pdf
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [Haskell] ANNOUNCE: haskell-src-exts 1.14.0
+1
When I worked on the font-lock support for haskell-mode, the irony
of trying to approximate the classification that the hugs/ghc/whatnot parser
was already doing wasn't lost on me. I still would like to tap into more
of the knowledge generated and lost in the compiler:
- A list of all tokens (source position, span, classification). Comments are
conventionally
treated as whitespace, but it's not very hard to capture them before dropping
them.
- All identifiers can be classified into def and use with enough lexical scope
information
to get from use to def and from def to all uses. Take this one step further
and type
information can be stored with the def.
- Tokens should be mapped to the underlying AST if possible, and vice versa.
I'm sure there's more, but with this, one could build an awesome editor, code
navigator.
I think it's possible, but I don't have time to work on it, alas. I'd like
suggestions as to
to realize this.
Tommy -- ponding hacking the parser combinators to keep this information.
On Aug 20, 2013, at 03:00 , Niklas Hambüchen wrote:
> On 20/08/13 18:19, Niklas Broberg wrote:
>> Sadly not - it's theoretically impossible. The fact that you can put
>> comments literally wherever, means that it's impossible to treat them as
>> nodes of the AST. E.g.
>>
>> f {- WHERE -} x = -- WOULD
>> -- THESE
>> do -- COMMENTS
>> a {- END -} <- g x -- UP
>> return {- ? -} a
>
> Oh, I see what you mean.
>
> I guess what I mean instead is:
>
> * A lex list that contains *everything*, including comments and white space
>
> * A full syntax tree of which each node points to (indexes) a position
> in the lex list to get the precise original position; comments in
> between two nodes can then be determined and more easily played with
> because they are between their positions in the lex list
>
> * An abstract syntax tree that has whitespace and comments discarded
> (what HSE has now)
>
> ___
> Haskell-Cafe mailing list
> Haskell-Cafe@haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [Haskell] ANNOUNCE: haskell-src-exts 1.14.0
This is not using haskell-src-exts, but the Haskell Refactorer has a
structure to keep a parallel tree of tokens indexed by SrcSpan, which
attempts to allocate comments to the appropriate point.
See
https://github.com/alanz/HaRe/blob/master/src/Language/Haskell/Refact/Utils/TokenUtils.hs.
It does not make use of the AST itself, so may be usable with
haskell-src-exts
Alan
On Tue, Aug 20, 2013 at 11:19 AM, Niklas Broberg
wrote:
> Hi Niklas,
>
> 1) My most desired feature would be a syntax tree that does not pluck
>> pluck comments out and make me treat them separately. It looks much
>> easier to me to have a fully descriptive tree and (filter . concatMap) /
>> traverse them out in some way than getting a list of comments and having
>> to insert them back in the right places myself.
>> Is that possible?
>>
>
> Sadly not - it's theoretically impossible. The fact that you can put
> comments literally wherever, means that it's impossible to treat them as
> nodes of the AST. E.g.
>
> f {- WHERE -} x = -- WOULD
> -- THESE
> do -- COMMENTS
> a {- END -} <- g x -- UP
> return {- ? -} a
>
> What would be theoretically possible is to define a restricted language
> that allows comments only in certain well-defined places (cf haddock), and
> ignores any others. That's a lot of work though, and it's not clear how big
> the gain is. :-\
>
> A different solution could be to improve the support, through better
> helper functions, for handling a syntax tree and a list of comments
> together. That's something I think could be worthwhile.
>
>
>> 2) Have you considered downloading the all-of-Hackage tarball and
>> running haskell-src-exts over it to get a benchmark of how much HSE can
>> already parse of the Haskell code out there?
>>
>
> Considered, yes. Done, no. Would love to see the results :-). The crew at
> OdHac (Roman, Erik, Simon) ensured that the current version handles all of
> 'base', which is a good start.
>
> Cheers, Niklas
>
> ___
> Haskell-Cafe mailing list
> Haskell-Cafe@haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [Haskell] ANNOUNCE: haskell-src-exts 1.14.0
On 20/08/13 11:02, JP Moresmau wrote: > BuildWrapper has some code that tries to link back the comments to the > declaration from the AST generated by haskell-src-exts and the comments. > See > https://github.com/JPMoresmau/BuildWrapper/blob/master/src/Language/Haskell/BuildWrapper/Src.hs. > The unit tests provide some samples: > https://github.com/JPMoresmau/BuildWrapper/blob/master/test/Language/Haskell/BuildWrapper/CMDTests.hs#L572-L638. > Maybe this can help you. > > JP It certainly look like I might be able to learn from this. Thank you. -- Mateusz K. 0x2ADA9A97.asc Description: application/pgp-keys ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [Haskell] ANNOUNCE: haskell-src-exts 1.14.0
On Tue, Aug 20, 2013 at 11:19 AM, Niklas Broberg wrote: > On Tue, Aug 20, 2013 at 10:48 AM, Niklas Hambüchen wrote: 2) Have you considered downloading the all-of-Hackage tarball and >> > running haskell-src-exts over it to get a benchmark of how much HSE can >> already parse of the Haskell code out there? >> > > Considered, yes. Done, no. Would love to see the results :-). The crew at > OdHac (Roman, Erik, Simon) ensured that the current version handles all of > 'base', which is a good start. > See: Nikolaos Bezirgiannis, Johan Jeuring and Sean Leather. Usage of Generic Programming on Hackage - Experience Report. WGP 2013. http://www.cs.uu.nl/research/techreps/repo/CS-2013/2013-014.pdf http://hackage.haskell.org/package/gpah Unfortunately, it seems we don't mention which version of haskell-src-exts is used for the article. But I'm certain it's 1.13.*, probably 1.13.5. Regards, Sean ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [Haskell] ANNOUNCE: haskell-src-exts 1.14.0
BuildWrapper has some code that tries to link back the comments to the declaration from the AST generated by haskell-src-exts and the comments. See https://github.com/JPMoresmau/BuildWrapper/blob/master/src/Language/Haskell/BuildWrapper/Src.hs. The unit tests provide some samples: https://github.com/JPMoresmau/BuildWrapper/blob/master/test/Language/Haskell/BuildWrapper/CMDTests.hs#L572-L638. Maybe this can help you. JP On Tue, Aug 20, 2013 at 11:21 AM, Mateusz Kowalczyk wrote: > On 20/08/13 09:48, Niklas Hambüchen wrote: > > Nice! > > > > I hope that haskell-suite will eventually become awesome and solve most > > of our automation-on-Haskell-code needs. > > > > Two questions: > > > > 1) My most desired feature would be a syntax tree that does not pluck > > pluck comments out and make me treat them separately. It looks much > > easier to me to have a fully descriptive tree and (filter . concatMap) / > > traverse them out in some way than getting a list of comments and having > > to insert them back in the right places myself. > > Is that possible? > > > +1 for this. There was a small discussion relevant to this on café > recently, if anyone is interested: > http://comments.gmane.org/gmane.comp.lang.haskell.cafe/106768 > > > 2) Have you considered downloading the all-of-Hackage tarball and > > running haskell-src-exts over it to get a benchmark of how much HSE can > > already parse of the Haskell code out there? > > > > Thanks! > > > > > -- > Mateusz K. > > ___ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe > > -- JP Moresmau http://jpmoresmau.blogspot.com/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [Haskell] ANNOUNCE: haskell-src-exts 1.14.0
On 20/08/13 18:19, Niklas Broberg wrote:
> Sadly not - it's theoretically impossible. The fact that you can put
> comments literally wherever, means that it's impossible to treat them as
> nodes of the AST. E.g.
>
> f {- WHERE -} x = -- WOULD
> -- THESE
> do -- COMMENTS
> a {- END -} <- g x -- UP
> return {- ? -} a
Oh, I see what you mean.
I guess what I mean instead is:
* A lex list that contains *everything*, including comments and white space
* A full syntax tree of which each node points to (indexes) a position
in the lex list to get the precise original position; comments in
between two nodes can then be determined and more easily played with
because they are between their positions in the lex list
* An abstract syntax tree that has whitespace and comments discarded
(what HSE has now)
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [Haskell] ANNOUNCE: haskell-src-exts 1.14.0
On 20/08/13 09:48, Niklas Hambüchen wrote: > Nice! > > I hope that haskell-suite will eventually become awesome and solve most > of our automation-on-Haskell-code needs. > > Two questions: > > 1) My most desired feature would be a syntax tree that does not pluck > pluck comments out and make me treat them separately. It looks much > easier to me to have a fully descriptive tree and (filter . concatMap) / > traverse them out in some way than getting a list of comments and having > to insert them back in the right places myself. > Is that possible? > +1 for this. There was a small discussion relevant to this on café recently, if anyone is interested: http://comments.gmane.org/gmane.comp.lang.haskell.cafe/106768 > 2) Have you considered downloading the all-of-Hackage tarball and > running haskell-src-exts over it to get a benchmark of how much HSE can > already parse of the Haskell code out there? > > Thanks! > -- Mateusz K. 0x2ADA9A97.asc Description: application/pgp-keys ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [Haskell] ANNOUNCE: haskell-src-exts 1.14.0
Hi Niklas,
1) My most desired feature would be a syntax tree that does not pluck
> pluck comments out and make me treat them separately. It looks much
> easier to me to have a fully descriptive tree and (filter . concatMap) /
> traverse them out in some way than getting a list of comments and having
> to insert them back in the right places myself.
> Is that possible?
>
Sadly not - it's theoretically impossible. The fact that you can put
comments literally wherever, means that it's impossible to treat them as
nodes of the AST. E.g.
f {- WHERE -} x = -- WOULD
-- THESE
do -- COMMENTS
a {- END -} <- g x -- UP
return {- ? -} a
What would be theoretically possible is to define a restricted language
that allows comments only in certain well-defined places (cf haddock), and
ignores any others. That's a lot of work though, and it's not clear how big
the gain is. :-\
A different solution could be to improve the support, through better helper
functions, for handling a syntax tree and a list of comments together.
That's something I think could be worthwhile.
> 2) Have you considered downloading the all-of-Hackage tarball and
> running haskell-src-exts over it to get a benchmark of how much HSE can
> already parse of the Haskell code out there?
>
Considered, yes. Done, no. Would love to see the results :-). The crew at
OdHac (Roman, Erik, Simon) ensured that the current version handles all of
'base', which is a good start.
Cheers, Niklas
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [Haskell] ANNOUNCE: haskell-src-exts 1.14.0
Nice! I hope that haskell-suite will eventually become awesome and solve most of our automation-on-Haskell-code needs. Two questions: 1) My most desired feature would be a syntax tree that does not pluck pluck comments out and make me treat them separately. It looks much easier to me to have a fully descriptive tree and (filter . concatMap) / traverse them out in some way than getting a list of comments and having to insert them back in the right places myself. Is that possible? 2) Have you considered downloading the all-of-Hackage tarball and running haskell-src-exts over it to get a benchmark of how much HSE can already parse of the Haskell code out there? Thanks! ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
