yes eclipse works by having partitioners, and scanners... its not really
parsing, but its meant to be fast and fault tolerant, so that the editor
knows where you editing, and what to highlight etc.. these are things that
parsers aren't too good at. I have heard rumours of people using antlr
grammars to scan documents in eclipse, but never seen anything concrete.
On 5/3/06, Russ Egan <[EMAIL PROTECTED]> wrote:
Interesting. That might not work for the IDE plugin though,
performance-wise. I think it relies on the fastest possible token lexing
to enable code highlighting. Intellij actually only invokes the parsing
phase when necessary. I think. I don't know, I'm might making this up
now ;).
If I get a jflex file that works, I'll post it, if anyone is curious.
On Wed, 03 May 2006 06:18:49 -0400, Michael Neale
<[EMAIL PROTECTED]> wrote:
> yes the code block is a problem. I am not 100% happy with what we have
at
> the moment.
> The problem is, as you say, identifying the code blocks to you don't lex
> them as if they are rules (this would be completely invisible to the
> user,
> except when it occasionally gets it wrong).
>
> My solution ( for 3.1) is to have a parser stage that actually does
> insert
> the "markers" as you say. It does this as it has lexer rules for where a
> code block may appear. The the main parser can have a lexer rule to read
> in
> a code block verbatim, just like a string, yes still have parser rules
> for
> the rule langauge elements. The leap I had to make is to understand the
> different between lexer and parser rules and how they behave and what
> their
> limitations are. As a code block could be *anything*, the only foolproof
> way
> is to have (or insert) markers to aid the lexing.
>
> Anyway, thats enough parsers 101 for today ;) I find it very
interesting,
> but I doubt others do.
>
> No idea on intelliJ, no doubt they probably have some cool ways of
> solving
> those problems. JFlex is GPL, so we can't touch it due to the GPL nature
> (generally we avoid LGPL as well, but its not as serious).
>
>
>
> On 5/3/06, Russ Egan <[EMAIL PROTECTED]> wrote:
>>
>> I'm not parsing the java blocks either. I'm just trying to recognize
>> what
>> is a java block. Once I recognize, I treat all the java stuff as a
>> string
>> token.
>>
>> Actually, I think intellij has a nifty way to correctly handle one
>> language embedded in another language, but I haven't figured out how it
>> works yet. I'm hoping I can just pass the java blocks off to idea's
>> java
>> lexer and parser, and get all the java code-completion,
>> error-highlighting, etc for free.
>>
>> I tried looking at the eclipse plugin for how it lexed and parsed, but
>> it
>> made my eyes cross. I have zero experience with
>> lexing/parsing/compiling,
>> so it's all Greek to me.
>>
>> On Tue, 02 May 2006 11:27:04 -0400, Mark Proctor <[EMAIL PROTECTED]
>
>> wrote:
>>
>> > In Drools 3.0 we actually don't parse the expressions and code
>> blocks.
>> > Instead we build up an AST tree of descrs - expressions and code
>> blocks
>> > are Strings. I then generate and compile classes for those
expressions
>> > and blocks. Although I believe Kris has a way to get jdt java
>> > intellisense and syntax highlighting within the code expressions and
>> > blocks - but thats not involving a full parse, I think.
>> >
>> > Mark
>> > Russ Egan wrote:
>> >> I guess that would save some time implementing the parser, but
>> there's
>> >> still the issue of recognizing the embedded java code in the drl
>> file.
>> >>
>> >> The lexing problem is just hard though, not impossible. JFlex has
>> >> these "modes", where each mode has it's own set of tokens, so as
long
>> >> as I can reliably figure out when I'm starting and stopping a block
>> of
>> >> java, I can switch to a java mode where I just accept anything. I'm
>> >> using counters to figure out when I hit the close-paren that marks
>> the
>> >> end of the java block (as opposed to a paren within in the java
>> >> block). Actually, the trickier part is figuring out where the java
>> >> block begins, especially in the LHS, since it can show up inside
>> >> equality expressions and expression constraints and eval blocks. I
>> end
>> >> up having to create a lot of JFlex modes to capture "ok, I'm in the
>> >> LHS. Now I'm in a column. Now I'm in the constraint. Ok, NOW the
>> >> left paren means 'start java'" etc. It's messy.
>> >>
>> >> It would be much easier if all java was bounded by a unique
character
>> >> (a character not used elsewhere in the drl syntax), like [] or {}.
>> >>
>> >> On Tue, 02 May 2006 08:33:03 -0400, Jérôme BERNARD
>> >> <[EMAIL PROTECTED]> wrote:
>> >>
>> >>> I've done a quick search on Google and found two interesting
things:
>> >>> - the homepage of JFlex (http://jflex.de/) where they talk
(although
>> >>> really briefly) about an integration with ANTLR,
>> >>> - a post
>> >>> (
>>
http://www.antlr.org:8080/pipermail/antlr-interest/2006-April/015974.html)
>> >>> on the ANTLR ML where someone did something quite similar.
>> >>>
>> >>> So a possible solution seems to be to "hack" the ANTLR grammar in
>> >>> order to use JFlex lexer instead of the ANTLR one.
>> >>> BTW a side effect could be potential speed improvement as some
>> >>> benchmarks "proves" JFlex lexer faster than the ANTLR one.
>> >>> Now maintenance might be a bit harder as a change in Drools grammar
>> >>> would have to be "replicated" in the ANTLR+JFlex one.
>> >>>
>> >>> Hope this helps.
>> >>>
>> >>> Regards,
>> >>> Jérôme.
>> >>>
>> >>>
>> >>> On 2 mai 06, at 01:56, Michael Neale wrote:
>> >>>
>> >>>> yes recognising blocks of code is immensely hard, as you kind of
>> have
>> >>>> to
>> >>>> expect anything in those blocks. I am planning in future to add in
>> a
>> >>>> "CDATA"
>> >>>> like feature to make lexing more seamless (just a small
improvement
>> >>>> that
>> >>>> means we can avoid keyword collissions).
>> >>>>
>> >>>> It sounds like jflex has done the easy part, and left the hard
part
>> >>>> up to
>> >>>> you ;) still, its interesting.
>> >>>>
>> >>>> On 5/2/06, Russ Egan <[EMAIL PROTECTED]> wrote:
>> >>>>>
>> >>>>> I started giving this a shot, by starting with their java script
>> >>>>> plugin.
>> >>>>> But I've found translating antlr to jflex is tricky: where antlr
>> >>>>> defines
>> >>>>> both the lexer and the parser, jflex is just a lexer. That makes
>> it
>> >>>>> tricky to recognize the embedded java code in the drl files. You
>> >>>>> need to
>> >>>>> figure out which blocks of text are java, and therefore shouldn't
>> be
>> >>>>> more
>> >>>>> granularly tokenized. And you need to figure that out by just
>> using
>> >>>>> token
>> >>>>> recognition.
>> >>>>>
>> >>>>> I don't have any experience with lexers or compilers (I'm not a
>> >>>>> compsci
>> >>>>> grad), so maybe it's not really has hard as it seems to me.
>> >>>>>
>> >>>>> The other drawback to intellij's support is that you can't
>> leverage
>> >>>>> a nice
>> >>>>> tool like antlr to handle parsing. You have to write the parsing
>> by
>> >>>>> hand. At least, that's how their javascript plugin works. It
>> might
>> >>>>> be
>> >>>>> possible to embed antlr in the plugin, and adapt it's output to
>> >>>>> IDEA's
>> >>>>> plugin API. Unfortunately, the plugin is expected to expose the
>> >>>>> intermediary lexer output directly to the IDE to enable
>> highlighting
>> >>>>> and
>> >>>>> such.
>> >>>>>
>> >>>>> On Mon, 01 May 2006 06:10:20 -0400, Michael Neale
>> >>>>> <[EMAIL PROTECTED]> wrote:
>> >>>>>
>> >>>>> > I would love to, but we get killed for even looking at
something
>> >>>>> non
>> >>>>> open
>> >>>>> > source sideways ;) as we are an antlr grammar, that would be
>> nice.
>> >>>>> >
>> >>>>> > On 5/1/06, Fabian Crabus <[EMAIL PROTECTED]> wrote:
>> >>>>> >>
>> >>>>> >> jepp...the meta programming system is not for mere mortals
>> (tried
>> >>>>> it,
>> >>>>> >> had
>> >>>>> >> a look
>> >>>>> >> at the hello world, didn't grasp a bit, uninstalled
>> >>>>> it)...nevertheless
>> >>>>> >> they offer
>> >>>>> >> custom language support via their plugin sdk. It's based on
>> some
>> >>>>> jflex
>> >>>>> >> magic, that
>> >>>>> >> can be adapted to antlr based languages- though it's not
really
>> >>>>> for
>> >>>>> >> the faint of heart.
>> >>>>> >> So I'd hoped that someone dug a little deeper...
>> >>>>> >>
>> >>>>> >> On 5/1/06, Michael Neale <[EMAIL PROTECTED]> wrote:
>> >>>>> >> > Not that I am aware of. Has been a few years since I have
>> used
>> >>>>> >> intelliJ,
>> >>>>> >> but
>> >>>>> >> > I am aware that they have some advanced concepts around
>> >>>>> >> meta-programming,
>> >>>>> >> > and making new languages, but they are probably not open
>> >>>>> source, so
>> >>>>> >> that
>> >>>>> >> may
>> >>>>> >> > put a dampener on things.
>> >>>>> >> >
>> >>>>> >> > On 5/1/06, Fabian Crabus <[EMAIL PROTECTED]> wrote:
>> >>>>> >> > >
>> >>>>> >> > > Hi,
>> >>>>> >> > >
>> >>>>> >> > > speaking of rule editors: is anyone working on an IntelliJ
>> >>>>> plugin?
>> >>>>> >> > > Something along the lines of the groovyJ plugin?
>> >>>>> >> > > I've started to look into IntelliJ's custom language
>> support
>> >>>>> but
>> >>>>> the
>> >>>>> >> > > documentation has -hmm- a potential to be more helpful ;)
>> >>>>> >> > >
>> >>>>> >> > > So has anyone done some work on something like this? Or
can
>> >>>>> >> > > anyone give me a pointer?
>> >>>>> >> > >
>> >>>>> >> > > Thanks,
>> >>>>> >> > > Fabian
>> >>>>> >> > >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> -- Using Opera's revolutionary e-mail client:
>> >>>>> http://www.opera.com/mail/
>> >>>>>
>> >>>>>
>> >>>
>> >>>
>> >>> -- Jérôme BERNARD,
>> >>> Kalixia, SARL.
>> >>> http://weblog.kalixia.com
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >> --Using Opera's revolutionary e-mail client:
>> http://www.opera.com/mail/
>> >>
>> >>
>> >>
>> >
>> >
>>
>>
>>
>> --
>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>>
>>
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
