[il-antlr-interest: 29011] Re: [antlr-interest] ANTLR v4 progress // switch fro v2.7.2
On 27/5/10 2:18 AM, "Terence Parr" wrote: Hi Terrence, > Just passing along an example HTML subset lexer/parser using ANTLR v4; thanks > to debugging and moral support from Oliver Zeigermann, we got the code > generation and runtime support working sufficiently to use the following > grammars. generate some really nice code indeed. You will note that, except > for the enhancement of the lexer modes, the grammars are backward compatible > with v3 :) Congratulation on progress. :) My question is. Currently we still use ANTLR 2.7.2. Was not time jump to v3 yet. What will be your advice: jump in nearest months to v3, and later this will move smooth to v4 also. Or just wait for v4 to avoid too big transfers? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29010] Re: [antlr-interest] ANTLR v4 progress
Way cool! Just curious...is the modal state variable automatically accessible
to the parser? Perhaps on a subchannel synced with the token stream? Sorry to
be tuning in late...always interested in new paths to semiosis. :)
Anyway...I need to take a look at girhub and think up a fun use.
Kyle
Sent from my Verizon Wireless BlackBerry
-Original Message-
From: Terence Parr
Date: Wed, 26 May 2010 16:18:28
To: antlr-interest@antlr.org interest
Subject: [antlr-interest] ANTLR v4 progress
Just passing along an example HTML subset lexer/parser using ANTLR v4; thanks
to debugging and moral support from Oliver Zeigermann, we got the code
generation and runtime support working sufficiently to use the following
grammars. generate some really nice code indeed. You will note that, except
for the enhancement of the lexer modes, the grammars are backward compatible
with v3 :)
I still have a long way to go, but it's looking more & more useful (only does
LL(1) code generation at this point).
Ter
---
lexer grammar HTMLLexer;
TAG_START : '<' {pushMode(INSIDE);} ;
COMMENT : '' {skip();} ;
TEXT : ~'<'+ ;
mode INSIDE;
TAG_STOP : '>' {popMode();} ;
END_TAG : '/' ID '>' {popMode();} ;
ID : ('A'..'Z'|'a'..'z'|'0'..'9'|'_'|'#')+ ;
EQ : '=' ;
STRING : '"' .* '"'
;
WS : ' '+ {skip();} ;
parser grammar HTMLParser;
options { tokenVocab=HTMLLexer; }
file : ( TAG_START (starttag | endtag) | TEXT)+ EOF ;
starttag : ID attr* TAG_STOP ;
attr : ID (EQ (ID|STRING))? ;
endtag
:END_TAG
;
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29009] [antlr-interest] ANTLR v4 progress
Just passing along an example HTML subset lexer/parser using ANTLR v4; thanks
to debugging and moral support from Oliver Zeigermann, we got the code
generation and runtime support working sufficiently to use the following
grammars. generate some really nice code indeed. You will note that, except
for the enhancement of the lexer modes, the grammars are backward compatible
with v3 :)
I still have a long way to go, but it's looking more & more useful (only does
LL(1) code generation at this point).
Ter
---
lexer grammar HTMLLexer;
TAG_START : '<' {pushMode(INSIDE);} ;
COMMENT : '' {skip();} ;
TEXT : ~'<'+ ;
mode INSIDE;
TAG_STOP : '>' {popMode();} ;
END_TAG : '/' ID '>' {popMode();} ;
ID : ('A'..'Z'|'a'..'z'|'0'..'9'|'_'|'#')+ ;
EQ : '=' ;
STRING : '"' .* '"'
;
WS : ' '+ {skip();} ;
parser grammar HTMLParser;
options { tokenVocab=HTMLLexer; }
file : ( TAG_START (starttag | endtag) | TEXT)+ EOF ;
starttag : ID attr* TAG_STOP ;
attr : ID (EQ (ID|STRING))? ;
endtag
:END_TAG
;
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29008] Re: [antlr-interest] Ambiguous grammar
Please see the FAQ:
http://www.antlr.org/wiki/display/ANTLR3/Lexer+grammar+for+floating+point%2C+dot%2C+range%2C+time+specs
Also note that there is a search engine:
http://antlr.markmail.org
For this group, and it is really good:
http://antlr.markmail.org/search/?q=range
Jim
> -Original Message-
> From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
> boun...@antlr.org] On Behalf Of Ondrej Krc-Jediny
> Sent: Wednesday, May 26, 2010 1:34 PM
> To: antlr-interest@antlr.org
> Subject: [antlr-interest] Ambiguous grammar
>
> Hello,
>
> I am new to ANTLR and I'd appreciate help with my problem. At first
> some grammar rules I am going to talk about:
>
> DIGIT: '0'..'9';
> LETTER: 'a'..'z'|'A'..'Z';
>
> REAL
> : (DIGIT)+ '.' (DIGIT)* EXPONENT?
> | '.' (DIGIT)+ EXPONENT?
> | (DIGIT)+ EXPONENT
> ;
>
> INTEGER: DIGIT+;
>
> IDENTIFIER: LETTER+;
>
> range_expr:
> expr ('..' expr)?;
>
> expr: expratom ('+' expratom)?;
>
> expratom: INTEGER | REAL | IDENTIFIER;
>
> This grammar allows '1.' and .1' as correct real numbers. Also, it
> allows range expression '1..2', which gets parsed incorrectly as two
> real numbers 1. and .2. Redefining token
> INTEGER: DIGIT '..'?;
> doesn't help, because then also valid range expression 'i + 1..2'. Is
> there any way how to parse the '..' correctly in all cases?
>
> Thank you very much,
>
> Ondrej.
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29007] [antlr-interest] Ambiguous grammar
Hello,
I am new to ANTLR and I'd appreciate help with my problem. At first
some grammar rules I am going to talk about:
DIGIT: '0'..'9';
LETTER: 'a'..'z'|'A'..'Z';
REAL
: (DIGIT)+ '.' (DIGIT)* EXPONENT?
| '.' (DIGIT)+ EXPONENT?
| (DIGIT)+ EXPONENT
;
INTEGER: DIGIT+;
IDENTIFIER: LETTER+;
range_expr:
expr ('..' expr)?;
expr: expratom ('+' expratom)?;
expratom: INTEGER | REAL | IDENTIFIER;
This grammar allows '1.' and .1' as correct real numbers. Also, it
allows range expression '1..2', which gets parsed incorrectly as two
real numbers 1. and .2. Redefining token
INTEGER: DIGIT '..'?;
doesn't help, because then also valid range expression 'i + 1..2'. Is
there any way how to parse the '..' correctly in all cases?
Thank you very much,
Ondrej.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29006] [antlr-interest] greedy subrule option idiom
Hello,
Following is a lexer rule to match quoted string that allows backslash
escape sequence.
STRING
:'"' ( options {greedy=false;} : ( ~ '\\' | '\\' . ) )* '"'
;
It seems to work. But if you put the '*' operator inside the subrule
like this:
STRING
:'"' ( options {greedy=false;} : ( ~ '\\' | '\\' . )* ) '"'
;
It eats up everything to EOF.
It's as if the greedy option applies to the ((subrule)*) instead of the
subrule itself, and only if the subrule is suffixed with '*' operator
(or with '+') externally (as in (subrule)*).
To my eyes, the second version seems the "correct" one.
Thoughts?
J
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29005] Re: [antlr-interest] Inexplicable RewriteEmptyStreamException in parser grammar
False alarm. There was some subtle abuse going on deep down in our setup of
the parser (our tree adaptor was creating error nodes incorrectly). Apologies!
Richard
Richard Thrippleton wrote:
> I'm using the Java target with ANTLR 3.2, and having a problem in a rule in
> my parser grammar that looks like this:
>
> "variableInit [ String name ]
> : ASSIGNMENT id=initializer
> -> ^(ASSIGNMENT IDENTIFIER[$ASSIGNMENT, $name] initializer)"
>
> In a particular case where the 'initializer' rule fails to match and issues a
> syntax error, I'm getting a RewriteEmptyStreamException that cites "rule
> initializer" popping up from a line that looks like this:
>
> "// 1151:5: -> ^( ASSIGNMENT IDENTIFIER[$ASSIGNMENT, $name] initializer )
> {
> {
> CommonTree root_1 = (CommonTree)adaptor.nil();
> root_1 = (CommonTree)adaptor.becomeRoot(stream_ASSIGNMENT.nextNode(),
> root_1);
> adaptor.addChild(root_1, (CommonTree)adaptor.create(IDENTIFIER,
> ASSIGNMENT179, name));
> adaptor.addChild(root_1, stream_initializer.nextTree());
> < This line
> adaptor.addChild(root_0, root_1);
> }
> }"
>
> In terms of error handling we're just using the default recovery (not
> overriding it with 'catch'/'rulecatch'). We've overridden reportError to
> pretty-print the RecognitionExceptions, but that's about it. Should I be
> needing to do anything special to cope with parsing failures of subrules, or
> should the default be sufficient to prevent these exceptions? Otherwise,
> where is the problem likely to lie - a defect elsewhere in my grammar, or in
> ANTLR?
>
> Cheers,
> Richard
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
\o/
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29004] Re: [antlr-interest] parsing rules
Your s: rule needs to set $st to something too or it will return an empty
template :-)
Jim
> -Original Message-
> From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
> boun...@antlr.org] On Behalf Of George Soom
> Sent: Wednesday, May 26, 2010 4:01 AM
> To: antlr-interest@antlr.org
> Subject: [antlr-interest] parsing rules
>
> Hi,
>
> I have to rewrite several files with mathematical text in them. Text
> substitutions are really simple (e.g. 'a=4' -> 'a:=4;' and similar).
> Input is in form:
>
> a=4
> b=3
> draw(a)
> draw(b)
>
>
> Needed output
> -
> a:=4;
> b:=3;
> draw(a, "constant arguments")
> draw(b, "constant arguments")
>
>
> The best option I have found is to use StringTemplate. At chapter 9.3
> in
> book 'The Definitive ANTLRD reference' has a suitable example.
> To explain my probelm I will add lines from two examples:
> Grammar:
>
> grammar T;
> options {output=template;}
> s : ID '=' INT ';' -> assign(x={$ID.text},y={$INT.text}) ;
> ---lexer part not pasted---
>
> Test.java
> --header, input, parser-lexer generation etc. not pasted---
> parser.setTemplateLib(templates); // give parser templates
> TParser.s_return r = parser.s(); // parse rule s
> StringTemplate output = r.getTemplate();
> System.out.println(output.toString());// emit translation
>
> This will work fine. But if I change rule s to
> s: assign+;
> assign: ID '=' INT ';' -> assign(x={$ID.text},y={$INT.text}) ;
>
> I have to call parse.assign() to get correct results. How can i still
> parse
> starting from rule 's' and get desired output?
>
> George
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29003] [antlr-interest] Using input.getSourceName() and INCLUDE processing
I have a JAVA language rewrite translator that adds some more modern control
constructs to an existing DSL. ANTLR has been invaluable.
I've provided "include" file support using the methods outlined in
http://www.antlr.org/wiki/pages/viewpage.action?pageId=557057 but I would like
to use the file name in the string template that is used to generate code:
| kw=KW_while {
whileLabelStack.push(new Label("while"));
}
l=string cond=string r=string
(s+=statement)+
KW_endwhile
-> template(lhs={$l.text},
operator={$cond.text.replaceAll("<", "\\<")},
rhs={$r.text},
s={$s},
file={input.getSourceName()},
line={$kw.line},
label={whileLabelStack.pop().getLabel()})
<<
; while
let ${_source_line} = ,
:
test _end:
; endwhile
goto :
_end:
>>
;
The problem is the attribute is always the name of the file that
contains the include directive. That is:
file1.cls:
--
#include file2.cls
while $01 < 1
let $01 = 1
endwhile
--
file2.cls:
--
while $01 < 2
let $01 = 2
endwhile
--
Translates to:
; while ${index} < 2
let ${_source_line} = file1.cls,1
L_0_while:
test L_0_while_end: ${index} < 2
let ${index} = 2
; endwhile
goto L_0_while:
L_0_while_end:
; while ${index} < 1
let ${_source_line} = file1.cls,3
L_1_while:
test L_1_while_end: ${index} < 1
let ${index} = 1
; endwhile
goto L_1_while:
L_1_while_end:
Notice that even though the line numbers are correct, the attribute
evaluates to file1.cls when processing file2.cls.
I've poured over the ANTLR Java runtime API documentation and it would appear
that input.getSourceName() should change when processing the included file.
Does anyone have an idea on what I'm doing wrong?
Thanks
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29002] [antlr-interest] parsing rules
Hi,
I have to rewrite several files with mathematical text in them. Text
substitutions are really simple (e.g. 'a=4' -> 'a:=4;' and similar).
Input is in form:
a=4
b=3
draw(a)
draw(b)
Needed output
-
a:=4;
b:=3;
draw(a, "constant arguments")
draw(b, "constant arguments")
The best option I have found is to use StringTemplate. At chapter 9.3 in
book 'The Definitive ANTLRD reference' has a suitable example.
To explain my probelm I will add lines from two examples:
Grammar:
grammar T;
options {output=template;}
s : ID '=' INT ';' -> assign(x={$ID.text},y={$INT.text}) ;
---lexer part not pasted---
Test.java
--header, input, parser-lexer generation etc. not pasted---
parser.setTemplateLib(templates); // give parser templates
TParser.s_return r = parser.s(); // parse rule s
StringTemplate output = r.getTemplate();
System.out.println(output.toString());// emit translation
This will work fine. But if I change rule s to
s: assign+;
assign: ID '=' INT ';' -> assign(x={$ID.text},y={$INT.text}) ;
I have to call parse.assign() to get correct results. How can i still parse
starting from rule 's' and get desired output?
George
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29001] Re: [antlr-interest] Error in AST
On 05/26/2010 11:18 AM, Nourchn Nor wrote:
> I wrote one grammar on AntlrWorks but one error is shown when I add same
> statement,
> Here is the error: "error(211): Essai.g:87:16: [fatal] rule predicate has
> non-LL(*) decision due to recursive rule invocations reachable from alts
> 5,6. Resolve by left-factoring or using syntactic predicates or using
> backtrack=true option"
> And if I have correct by backtrack=true, the AST of every rule in the syntax
> Diagram of Antlworks is consisted by { synpred1_Essai }? With Essai is the
> name of grammar.
> This is an error this { synpred1_Essai }? ? what can I have to correct
> him?
> please help me
Have you read the book? It explains LL(*) in detail, and what you have to
do to deal with things like this.
http://www.pragprog.com/titles/tpantlr/the-definitive-antlr-reference
Andrew.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 29000] [antlr-interest] Error in AST
I wrote one grammar on AntlrWorks but one error is shown when I add same
statement,
Here is the error: "error(211): Essai.g:87:16: [fatal] rule predicate has
non-LL(*) decision due to recursive rule invocations reachable from alts
5,6. Resolve by left-factoring or using syntactic predicates or using
backtrack=true option"
And if I have correct by backtrack=true, the AST of every rule in the syntax
Diagram of Antlworks is consisted by { synpred1_Essai }? With Essai is the
name of grammar.
This is an error this { synpred1_Essai }? ? what can I have to correct
him?
please help me
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
