yes the code block is a problem. I am not 100% happy with what we have at
the moment.
The problem is, as you say, identifying the code blocks to you don't lex
them as if they are rules (this would be completely invisible to the user,
except when it occasionally gets it wrong).
My solution ( for 3.1) is to have a parser stage that actually does insert
the "markers" as you say. It does this as it has lexer rules for where a
code block may appear. The the main parser can have a lexer rule to read in
a code block verbatim, just like a string, yes still have parser rules for
the rule langauge elements. The leap I had to make is to understand the
different between lexer and parser rules and how they behave and what their
limitations are. As a code block could be *anything*, the only foolproof way
is to have (or insert) markers to aid the lexing.
Anyway, thats enough parsers 101 for today ;) I find it very interesting,
but I doubt others do.
No idea on intelliJ, no doubt they probably have some cool ways of solving
those problems. JFlex is GPL, so we can't touch it due to the GPL nature
(generally we avoid LGPL as well, but its not as serious).
On 5/3/06, Russ Egan <[EMAIL PROTECTED]> wrote:
I'm not parsing the java blocks either. I'm just trying to recognize what
is a java block. Once I recognize, I treat all the java stuff as a string
token.
Actually, I think intellij has a nifty way to correctly handle one
language embedded in another language, but I haven't figured out how it
works yet. I'm hoping I can just pass the java blocks off to idea's java
lexer and parser, and get all the java code-completion,
error-highlighting, etc for free.
I tried looking at the eclipse plugin for how it lexed and parsed, but it
made my eyes cross. I have zero experience with lexing/parsing/compiling,
so it's all Greek to me.
On Tue, 02 May 2006 11:27:04 -0400, Mark Proctor <[EMAIL PROTECTED]>
wrote:
> In Drools 3.0 we actually don't parse the expressions and code blocks.
> Instead we build up an AST tree of descrs - expressions and code blocks
> are Strings. I then generate and compile classes for those expressions
> and blocks. Although I believe Kris has a way to get jdt java
> intellisense and syntax highlighting within the code expressions and
> blocks - but thats not involving a full parse, I think.
>
> Mark
> Russ Egan wrote:
>> I guess that would save some time implementing the parser, but there's
>> still the issue of recognizing the embedded java code in the drl file.
>>
>> The lexing problem is just hard though, not impossible. JFlex has
>> these "modes", where each mode has it's own set of tokens, so as long
>> as I can reliably figure out when I'm starting and stopping a block of
>> java, I can switch to a java mode where I just accept anything. I'm
>> using counters to figure out when I hit the close-paren that marks the
>> end of the java block (as opposed to a paren within in the java
>> block). Actually, the trickier part is figuring out where the java
>> block begins, especially in the LHS, since it can show up inside
>> equality expressions and expression constraints and eval blocks. I end
>> up having to create a lot of JFlex modes to capture "ok, I'm in the
>> LHS. Now I'm in a column. Now I'm in the constraint. Ok, NOW the
>> left paren means 'start java'" etc. It's messy.
>>
>> It would be much easier if all java was bounded by a unique character
>> (a character not used elsewhere in the drl syntax), like [] or {}.
>>
>> On Tue, 02 May 2006 08:33:03 -0400, Jérôme BERNARD
>> <[EMAIL PROTECTED]> wrote:
>>
>>> I've done a quick search on Google and found two interesting things:
>>> - the homepage of JFlex (http://jflex.de/) where they talk (although
>>> really briefly) about an integration with ANTLR,
>>> - a post
>>> (
http://www.antlr.org:8080/pipermail/antlr-interest/2006-April/015974.html)
>>> on the ANTLR ML where someone did something quite similar.
>>>
>>> So a possible solution seems to be to "hack" the ANTLR grammar in
>>> order to use JFlex lexer instead of the ANTLR one.
>>> BTW a side effect could be potential speed improvement as some
>>> benchmarks "proves" JFlex lexer faster than the ANTLR one.
>>> Now maintenance might be a bit harder as a change in Drools grammar
>>> would have to be "replicated" in the ANTLR+JFlex one.
>>>
>>> Hope this helps.
>>>
>>> Regards,
>>> Jérôme.
>>>
>>>
>>> On 2 mai 06, at 01:56, Michael Neale wrote:
>>>
>>>> yes recognising blocks of code is immensely hard, as you kind of have
>>>> to
>>>> expect anything in those blocks. I am planning in future to add in a
>>>> "CDATA"
>>>> like feature to make lexing more seamless (just a small improvement
>>>> that
>>>> means we can avoid keyword collissions).
>>>>
>>>> It sounds like jflex has done the easy part, and left the hard part
>>>> up to
>>>> you ;) still, its interesting.
>>>>
>>>> On 5/2/06, Russ Egan <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>> I started giving this a shot, by starting with their java script
>>>>> plugin.
>>>>> But I've found translating antlr to jflex is tricky: where antlr
>>>>> defines
>>>>> both the lexer and the parser, jflex is just a lexer. That makes it
>>>>> tricky to recognize the embedded java code in the drl files. You
>>>>> need to
>>>>> figure out which blocks of text are java, and therefore shouldn't be
>>>>> more
>>>>> granularly tokenized. And you need to figure that out by just using
>>>>> token
>>>>> recognition.
>>>>>
>>>>> I don't have any experience with lexers or compilers (I'm not a
>>>>> compsci
>>>>> grad), so maybe it's not really has hard as it seems to me.
>>>>>
>>>>> The other drawback to intellij's support is that you can't leverage
>>>>> a nice
>>>>> tool like antlr to handle parsing. You have to write the parsing by
>>>>> hand. At least, that's how their javascript plugin works. It might
>>>>> be
>>>>> possible to embed antlr in the plugin, and adapt it's output to
>>>>> IDEA's
>>>>> plugin API. Unfortunately, the plugin is expected to expose the
>>>>> intermediary lexer output directly to the IDE to enable highlighting
>>>>> and
>>>>> such.
>>>>>
>>>>> On Mon, 01 May 2006 06:10:20 -0400, Michael Neale
>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>> > I would love to, but we get killed for even looking at something
>>>>> non
>>>>> open
>>>>> > source sideways ;) as we are an antlr grammar, that would be nice.
>>>>> >
>>>>> > On 5/1/06, Fabian Crabus <[EMAIL PROTECTED]> wrote:
>>>>> >>
>>>>> >> jepp...the meta programming system is not for mere mortals (tried
>>>>> it,
>>>>> >> had
>>>>> >> a look
>>>>> >> at the hello world, didn't grasp a bit, uninstalled
>>>>> it)...nevertheless
>>>>> >> they offer
>>>>> >> custom language support via their plugin sdk. It's based on some
>>>>> jflex
>>>>> >> magic, that
>>>>> >> can be adapted to antlr based languages- though it's not really
>>>>> for
>>>>> >> the faint of heart.
>>>>> >> So I'd hoped that someone dug a little deeper...
>>>>> >>
>>>>> >> On 5/1/06, Michael Neale <[EMAIL PROTECTED]> wrote:
>>>>> >> > Not that I am aware of. Has been a few years since I have used
>>>>> >> intelliJ,
>>>>> >> but
>>>>> >> > I am aware that they have some advanced concepts around
>>>>> >> meta-programming,
>>>>> >> > and making new languages, but they are probably not open
>>>>> source, so
>>>>> >> that
>>>>> >> may
>>>>> >> > put a dampener on things.
>>>>> >> >
>>>>> >> > On 5/1/06, Fabian Crabus <[EMAIL PROTECTED]> wrote:
>>>>> >> > >
>>>>> >> > > Hi,
>>>>> >> > >
>>>>> >> > > speaking of rule editors: is anyone working on an IntelliJ
>>>>> plugin?
>>>>> >> > > Something along the lines of the groovyJ plugin?
>>>>> >> > > I've started to look into IntelliJ's custom language support
>>>>> but
>>>>> the
>>>>> >> > > documentation has -hmm- a potential to be more helpful ;)
>>>>> >> > >
>>>>> >> > > So has anyone done some work on something like this? Or can
>>>>> >> > > anyone give me a pointer?
>>>>> >> > >
>>>>> >> > > Thanks,
>>>>> >> > > Fabian
>>>>> >> > >
>>>>> >> >
>>>>> >> >
>>>>> >>
>>>>>
>>>>>
>>>>>
>>>>> -- Using Opera's revolutionary e-mail client:
>>>>> http://www.opera.com/mail/
>>>>>
>>>>>
>>>
>>>
>>> -- Jérôme BERNARD,
>>> Kalixia, SARL.
>>> http://weblog.kalixia.com
>>>
>>>
>>>
>>
>>
>>
>> --Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>>
>>
>>
>
>
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
