Hi Kaspar! Thanks so much for the response and for bringing the discussion
to the mailing list! Definitely the right thing to do in this case.
I get what you're saying about Transforms. I sensed it before and resisted
it, but now I really can't deny it. :) This is what I should be doing:
rule(:inline => '_', :content => sequence(:c)) { RedCloth::StrongElement(c)
}
rule(:inline => '_', :content => sequence(:c), :attributes => sequence(:a))
{ RedCloth::StrongElement(c, :attributes => a) }
etc.
...and then going from these AST classes to the desired output. It seemed
like the right way to go initially (and was what I was doing in Treetop) but
I dismissed it because it added an extra step. All those instances; it would
have to be slow, right? :-) Well, I should have tried it and then dealt with
speed issues as needed. At the rate my transform's complexity was
increasing, I think I was going to end up there anyway.
Thanks for taking the time to explain it to me. I can already see how much
cleaner this will make things: In the specs, rather than saying it should
parse(...).as(nasty_complex_hash), I could skip straight to
parse(...).should == StrongElement(...). The AST specs will be much cleaner
too.
So now I get it: parsing and transformation are two parts to the same
operation, you could say, and the intermediate data structure is sort of
emergent. It's internal and not something to worry about or coerce too much.
That's certainly liberating.
It would be easy to blame my misunderstanding on the documentation, but I
won't because the docs + examples are ridiculously awesome! They've answered
all my questions without bothering you fine folks. I might have figured this
out on my own too had I been able to resist the premature optimization
reflex. Of the examples in the examples directory, though, the majority
generate their output within the Transform. Only string_parser.rb defines an
AST.
So, Kaspar, perhaps in the documentation you'll find a way to downplay the
context of Transforms and communicate that in Parslet, parsing and
transformations go together like lexing and parsing, no? What you just told
me would be a great start:
Transformations are good for one thing: Getting out of the horrible
> Hash/Array/Slice mixture into the realm of your own AST classes easily.
> Usually, no state is associated with that operation.
Thanks again! Always a pleasure,
Jason
On Sun, May 8, 2011 at 6:51 AM, Kaspar Schiess <[email protected]> wrote:
> (This post answers pull request 37[1] on github. I think this is the
> better medium for discussing this)
>
> Hi Jason,
>
> I was/am aware of the lack of consistency in the execution environment
> for parslet's transformation blocks. That's on purpose. Let me explain:
>
> Unlike parsers (< Parslet::Parser), transformations are classes in their
> own right that have state and behaviour. Although the elements of a
> transformation are also called #rule, (which is unfortunate), they act
> very differently. My LISP loving heart wants transformations to be
> stateless, purely functional.
>
> I designed the transformations to be good for one thing: Getting out of
> the horrible Hash/Array/Slice mixture into the realm of your own AST
> classes easily. I think they fulfill that purpose well. Usually, no
> state is associated with that operation.
>
> To you it (going into an AST first, then doing work) may look like an
> added step with no rationale; I can see that and will try to improve
> transformations for people who want to use them to produce results
> directly. However. Your pull request encourages users of parslet to keep
> state in the transformation instance directly, which strikes me as very
> bad OOP - it violates the single responsibility principle.
>
> Having state in the transformation will also make it non-trivial to
> update to new parslet versions - I might use @doc in Parslet::Transform
> in the future, leaving you with broken tests.
>
> But let me propose something that might work for both of us: How about
> injecting context during the apply call?
>
> transform = Parslet::Transform.new do
> rule(...) { doc.insert(...) } # doc is defined here
> rule(...) { |d| d[:doc].insert(...) } # and also here
> end
>
> transform.apply(tree, :doc => MyDocument.new)
>
> My hope is to leave the execution context of transformations officially
> undefined, providing for everything needed for nice code without having
> to anchor them. If I find a way to inject local variables without the
> Context hack, I'll eliminate it, opting for definition scope everywhere.
> But I have currently no idea on how to do that. But the above change
> would be trivial.
>
> As an aside: I think implementation discussions and feature requests
> belong to the mailing list - more people watching this. A pull request
> is not a discussion of possibilities, it makes me say 'yes' or 'no',
> hard to strike a note between. Instead of giving you a 'no', I want to
> say 'yes, but how would you like this?'. Github doesn't let me do that
> very well.
>
> How about it?
> regards
> kaspar
>
> [1] https://github.com/kschiess/parslet/pull/37
>
>
