Interesting. That might not work for the IDE plugin though,
performance-wise. I think it relies on the fastest possible token
lexing
to enable code highlighting. Intellij actually only invokes the parsing
phase when necessary. I think. I don't know, I'm might making this up
now ;).
If I get a jflex file that works, I'll post it, if anyone is curious.
On Wed, 03 May 2006 06:18:49 -0400, Michael Neale
<[EMAIL PROTECTED]> wrote:
> yes the code block is a problem. I am not 100% happy with what we have
at
> the moment.
> The problem is, as you say, identifying the code blocks to you don't
lex
> them as if they are rules (this would be completely invisible to the
> user,
> except when it occasionally gets it wrong).
>
> My solution ( for 3.1) is to have a parser stage that actually does
> insert
> the "markers" as you say. It does this as it has lexer rules for
where a
> code block may appear. The the main parser can have a lexer rule to
read
> in
> a code block verbatim, just like a string, yes still have parser rules
> for
> the rule langauge elements. The leap I had to make is to understand
the
> different between lexer and parser rules and how they behave and what
> their
> limitations are. As a code block could be *anything*, the only
foolproof
> way
> is to have (or insert) markers to aid the lexing.
>
> Anyway, thats enough parsers 101 for today ;) I find it very
interesting,
> but I doubt others do.
>
> No idea on intelliJ, no doubt they probably have some cool ways of
> solving
> those problems. JFlex is GPL, so we can't touch it due to the GPL
nature
> (generally we avoid LGPL as well, but its not as serious).
>
>
>
> On 5/3/06, Russ Egan <[EMAIL PROTECTED]> wrote:
>>
>> I'm not parsing the java blocks either. I'm just trying to recognize
>> what
>> is a java block. Once I recognize, I treat all the java stuff as a
>> string
>> token.
>>
>> Actually, I think intellij has a nifty way to correctly handle one
>> language embedded in another language, but I haven't figured out how
it
>> works yet. I'm hoping I can just pass the java blocks off to idea's
>> java
>> lexer and parser, and get all the java code-completion,
>> error-highlighting, etc for free.
>>
>> I tried looking at the eclipse plugin for how it lexed and parsed,
but
>> it
>> made my eyes cross. I have zero experience with
>> lexing/parsing/compiling,
>> so it's all Greek to me.
>>
>> On Tue, 02 May 2006 11:27:04 -0400, Mark Proctor
<[EMAIL PROTECTED]
>
>> wrote:
>>
>> > In Drools 3.0 we actually don't parse the expressions and code
>> blocks.
>> > Instead we build up an AST tree of descrs - expressions and code
>> blocks
>> > are Strings. I then generate and compile classes for those
expressions
>> > and blocks. Although I believe Kris has a way to get jdt java
>> > intellisense and syntax highlighting within the code expressions
and
>> > blocks - but thats not involving a full parse, I think.
>> >
>> > Mark
>> > Russ Egan wrote:
>> >> I guess that would save some time implementing the parser, but
>> there's
>> >> still the issue of recognizing the embedded java code in the drl
>> file.
>> >>
>> >> The lexing problem is just hard though, not impossible. JFlex has
>> >> these "modes", where each mode has it's own set of tokens, so as
long
>> >> as I can reliably figure out when I'm starting and stopping a
block
>> of
>> >> java, I can switch to a java mode where I just accept anything.
I'm
>> >> using counters to figure out when I hit the close-paren that marks
>> the
>> >> end of the java block (as opposed to a paren within in the java
>> >> block). Actually, the trickier part is figuring out where the
java
>> >> block begins, especially in the LHS, since it can show up inside
>> >> equality expressions and expression constraints and eval blocks.
I
>> end
>> >> up having to create a lot of JFlex modes to capture "ok, I'm in
the
>> >> LHS. Now I'm in a column. Now I'm in the constraint. Ok, NOW
the
>> >> left paren means 'start java'" etc. It's messy.
>> >>
>> >> It would be much easier if all java was bounded by a unique
character
>> >> (a character not used elsewhere in the drl syntax), like [] or {}.
>> >>
>> >> On Tue, 02 May 2006 08:33:03 -0400, Jérôme BERNARD
>> >> <[EMAIL PROTECTED]> wrote:
>> >>
>> >>> I've done a quick search on Google and found two interesting
things:
>> >>> - the homepage of JFlex (http://jflex.de/) where they talk
(although
>> >>> really briefly) about an integration with ANTLR,
>> >>> - a post
>> >>> (
>>
http://www.antlr.org:8080/pipermail/antlr-interest/2006-April/015974.html)
>> >>> on the ANTLR ML where someone did something quite similar.
>> >>>
>> >>> So a possible solution seems to be to "hack" the ANTLR grammar in
>> >>> order to use JFlex lexer instead of the ANTLR one.
>> >>> BTW a side effect could be potential speed improvement as some
>> >>> benchmarks "proves" JFlex lexer faster than the ANTLR one.
>> >>> Now maintenance might be a bit harder as a change in Drools
grammar
>> >>> would have to be "replicated" in the ANTLR+JFlex one.
>> >>>
>> >>> Hope this helps.
>> >>>
>> >>> Regards,
>> >>> Jérôme.
>> >>>
>> >>>
>> >>> On 2 mai 06, at 01:56, Michael Neale wrote:
>> >>>
>> >>>> yes recognising blocks of code is immensely hard, as you kind of
>> have
>> >>>> to
>> >>>> expect anything in those blocks. I am planning in future to add
in
>> a
>> >>>> "CDATA"
>> >>>> like feature to make lexing more seamless (just a small
improvement
>> >>>> that
>> >>>> means we can avoid keyword collissions).
>> >>>>
>> >>>> It sounds like jflex has done the easy part, and left the hard
part
>> >>>> up to
>> >>>> you ;) still, its interesting.
>> >>>>
>> >>>> On 5/2/06, Russ Egan <[EMAIL PROTECTED]> wrote:
>> >>>>>
>> >>>>> I started giving this a shot, by starting with their java
script
>> >>>>> plugin.
>> >>>>> But I've found translating antlr to jflex is tricky: where
antlr
>> >>>>> defines
>> >>>>> both the lexer and the parser, jflex is just a lexer. That
makes
>> it
>> >>>>> tricky to recognize the embedded java code in the drl files.
You
>> >>>>> need to
>> >>>>> figure out which blocks of text are java, and therefore
shouldn't
>> be
>> >>>>> more
>> >>>>> granularly tokenized. And you need to figure that out by just
>> using
>> >>>>> token
>> >>>>> recognition.
>> >>>>>
>> >>>>> I don't have any experience with lexers or compilers (I'm not a
>> >>>>> compsci
>> >>>>> grad), so maybe it's not really has hard as it seems to me.
>> >>>>>
>> >>>>> The other drawback to intellij's support is that you can't
>> leverage
>> >>>>> a nice
>> >>>>> tool like antlr to handle parsing. You have to write the
parsing
>> by
>> >>>>> hand. At least, that's how their javascript plugin works. It
>> might
>> >>>>> be
>> >>>>> possible to embed antlr in the plugin, and adapt it's output to
>> >>>>> IDEA's
>> >>>>> plugin API. Unfortunately, the plugin is expected to expose
the
>> >>>>> intermediary lexer output directly to the IDE to enable
>> highlighting
>> >>>>> and
>> >>>>> such.
>> >>>>>
>> >>>>> On Mon, 01 May 2006 06:10:20 -0400, Michael Neale
>> >>>>> <[EMAIL PROTECTED]> wrote:
>> >>>>>
>> >>>>> > I would love to, but we get killed for even looking at
something
>> >>>>> non
>> >>>>> open
>> >>>>> > source sideways ;) as we are an antlr grammar, that would be
>> nice.
>> >>>>> >
>> >>>>> > On 5/1/06, Fabian Crabus <[EMAIL PROTECTED]> wrote:
>> >>>>> >>
>> >>>>> >> jepp...the meta programming system is not for mere mortals
>> (tried
>> >>>>> it,
>> >>>>> >> had
>> >>>>> >> a look
>> >>>>> >> at the hello world, didn't grasp a bit, uninstalled
>> >>>>> it)...nevertheless
>> >>>>> >> they offer
>> >>>>> >> custom language support via their plugin sdk. It's based on
>> some
>> >>>>> jflex
>> >>>>> >> magic, that
>> >>>>> >> can be adapted to antlr based languages- though it's not
really
>> >>>>> for
>> >>>>> >> the faint of heart.
>> >>>>> >> So I'd hoped that someone dug a little deeper...
>> >>>>> >>
>> >>>>> >> On 5/1/06, Michael Neale <[EMAIL PROTECTED]> wrote:
>> >>>>> >> > Not that I am aware of. Has been a few years since I have
>> used
>> >>>>> >> intelliJ,
>> >>>>> >> but
>> >>>>> >> > I am aware that they have some advanced concepts around
>> >>>>> >> meta-programming,
>> >>>>> >> > and making new languages, but they are probably not open
>> >>>>> source, so
>> >>>>> >> that
>> >>>>> >> may
>> >>>>> >> > put a dampener on things.
>> >>>>> >> >
>> >>>>> >> > On 5/1/06, Fabian Crabus <[EMAIL PROTECTED]> wrote:
>> >>>>> >> > >
>> >>>>> >> > > Hi,
>> >>>>> >> > >
>> >>>>> >> > > speaking of rule editors: is anyone working on an
IntelliJ
>> >>>>> plugin?
>> >>>>> >> > > Something along the lines of the groovyJ plugin?
>> >>>>> >> > > I've started to look into IntelliJ's custom language
>> support
>> >>>>> but
>> >>>>> the
>> >>>>> >> > > documentation has -hmm- a potential to be more helpful
;)
>> >>>>> >> > >
>> >>>>> >> > > So has anyone done some work on something like this? Or
can
>> >>>>> >> > > anyone give me a pointer?
>> >>>>> >> > >
>> >>>>> >> > > Thanks,
>> >>>>> >> > > Fabian
>> >>>>> >> > >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> -- Using Opera's revolutionary e-mail client:
>> >>>>> http://www.opera.com/mail/
>> >>>>>
>> >>>>>
>> >>>
>> >>>
>> >>> -- Jérôme BERNARD,
>> >>> Kalixia, SARL.
>> >>> http://weblog.kalixia.com
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >> --Using Opera's revolutionary e-mail client:
>> http://www.opera.com/mail/
>> >>
>> >>
>> >>
>> >
>> >
>>
>>
>>
>> --
>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>>
>>
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
