[il-antlr-interest: 34962] Re: [antlr-interest] Failure to ignore newline
On Fri, Nov 18, 2011 at 7:55 AM, David Riddle wrote:
> Hi -
>
> This should be a very simple thing - I'm attempting to have my grammar hide
> newline, carriage returns, etc. However, every concievable form of a
> grammar that attempts to skip over these things or send them to the hidden
> channel seems to fail for me. Here's a very basic example:
>
> grammar Test;
>
> prog: ID+;
>
> ID: 'a'..'z'+;
>
> WS: '\n'+ {$channel=HIDDEN;};
>
> // Input: a \n b
> // Output: a n b
>
I'm guessing that's no 0xA (new line char) in your input, but a backslash
followed by a 'n'.
Regards,
Bart.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34963] [antlr-interest] String concatenation expression rule
I am writing a grammar for a fairly complex expression language, and in
particular I need to support string concatenation which is performed simply
by separating string literals with a space; and which automatically converts
other expressions to a string if needed to concatenate:
"a" "b" -> "ab"
2+3 "mm" -> "5mm"
I suspect I could use predicates to write a rule like this:
concatExpression
:( expression | STRING_LITERAL )+ { apply only if at least
one of the elements is a string literal }?
Is there a way to achieve this? The alternative formulations I can think of
are pretty messy...
Thanks!
--
View this message in context:
http://antlr.1301665.n2.nabble.com/String-concatenation-expression-rule-tp7007921p7007921.html
Sent from the ANTLR mailing list archive at Nabble.com.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34964] Re: [antlr-interest] String concatenation expression rule
On Fri, Nov 18, 2011 at 12:39 PM, franck102 wrote:
> I am writing a grammar for a fairly complex expression language, and in
> particular I need to support string concatenation which is performed simply
> by separating string literals with a space; and which automatically
> converts
> other expressions to a string if needed to concatenate:
> "a" "b" -> "ab"
> 2+3 "mm" -> "5mm"
>
> I suspect I could use predicates to write a rule like this:
>
> concatExpression
>:( expression | STRING_LITERAL )+ { apply only if at least
> one of the elements is a string literal }?
>
> Is there a way to achieve this? The alternative formulations I can think of
> are pretty messy...
>
>
As far as I understand it, you don't need any predicate. I see a
concat-expression has a lower precedence than addition, in which case this
could do the trick:
grammar T;
options {
output=AST;
}
tokens {
ROOT;
CONCAT;
}
parse
: (expression ';')* EOF -> ^(ROOT expression*)
;
expression
: (add -> add) (add+ -> ^(CONCAT add+))?
;
add
: atom (('+' | '-')^ atom)*
;
atom
: Number
| String
| '(' expression ')' -> expression
;
Number : '0'..'9'+ ('.' '0'..'9'+)?;
String : '"' ~'"'* '"';
Space : ' ' {skip();};
You can test it with the following class:
import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.*;
public class Main {
public static void main(String[] args) throws Exception {
String src = "42 - 2; 2 + 3 \"mm\"; \"a\" \"b\" 4-3-2 \"c\"; \"pi = \"
3.14159;";
TLexer lexer = new TLexer(new ANTLRStringStream(src));
TParser parser = new TParser(new CommonTokenStream(lexer));
CommonTree root = (CommonTree)parser.parse().getTree(); ;
System.out.println(new DOTTreeGenerator().toDOT(root));
}
}
Regards,
Bart.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34965] Re: [antlr-interest] confused about rendering all String Templates...
ahhh... just updated my grammar to output templates... you're right, once you explained it I knew what to look for in other examples I had worked through. Now when I'm calling toString on the template that was returned I'm getting everything that I accumulated while walking my AST. Now currently I'm not saving much in the way of cleaning up my tree parser, but now that I know what to expect wrt templates I can look for ways to refactor it and hopefully improve it! thanks again for the insight, left to my own devices I may have never stumbled across that =D Morgan On Thu, Nov 17, 2011 at 12:50 PM, Voelkel, Andy wrote: > It's very confusing trying to read the detailed documentation on the > StringTemplate site until you get the basic idea, but keep at it. A key thing > for me was realizing that the objects embedded as "parameters" in a string > template can be just about anything that can be rendered to a string, or > collections of things that can be rendered to a string, or StringTemplates > themselves. That is what makes the recursion possible, and makes them so > powerful. Do you have "The Definitive Guide to Antlr"? That's handy, although > it is still a leap from his examples to understanding the terse online > documentation. > > - Andy > > -Original Message- > From: morg...@gmail.com [mailto:morg...@gmail.com] On Behalf Of Morgan Jones > Sent: Thursday, November 17, 2011 9:43 AM > To: Voelkel, Andy > Cc: antlr-interest@antlr.org > Subject: Re: [antlr-interest] confused about rendering all String Templates... > > Andy, that does help... thanks for replying. > as I read about StringTemplates today I gathered they were capable of much > more than I had initially realized... > I'll go back and try to track down some examples of the nested String > Templates you spoke of. > > Maybe I'll luck out and someone will point me to a concrete example. ;) > > Morgan > > On Thu, Nov 17, 2011 at 12:14 PM, Voelkel, Andy > wrote: >> Hi Morgan, >> >> Well, I'm a newbie too, but I can tell you how I understand and use >> StringTemplates. For me, the end result of walking a tree that produces >> StringTemplates is a StringTemplate which contains elements which are >> themselves StringTemplates (or lists of String Templates), which contains >> elements which are themselves StringTemplates (or lists of String >> Templates), and so on. At this point no "rendering" to actual strings has >> been done. After the giant StringTemplate amalgam is created, rendering the >> top level StringTemplate will render all the sub-StringTemplate in turn, all >> the way down the tree. >> >> In a way you have created a different sort of StringTemplate tree during the >> tree walk, but it is not the AST itself. >> >> So as you are walking a tree, you are taking the StringTemplates returned by >> rules and putting them into StringTemplates of the containing rules, and the >> top level rule returns the top level StringTemplate. >> >> I hope this is clear, and the experts can correct my >> misunderstandings, >> >> - Andy >> >> -Original Message- >> From: antlr-interest-boun...@antlr.org >> [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Morgan Jones >> Sent: Thursday, November 17, 2011 8:39 AM >> To: antlr-interest@antlr.org >> Subject: [antlr-interest] confused about rendering all String Templates... >> >> I've gotten a lexer, parser (that outputs AST) and a tree parser work. >> >> Currently the tree parser contain little code snippets that include >> System.out.println to print my translated file. >> >> Not very pretty, but it works. >> >> My next step was to switch to StringTemplates, because I thought it would >> help me clean up the tree parser. >> So far my initial efforts haven't yielded much. >> >> After reading TP's books and (this morning) about 40 pages of interest >> messages about templates I'm beginning to think I might be misunderstanding >> the use of String Templates... >> >> I thought I could include a template and after walking my tree I could call >> getTemplate on my return object and then toString to have it dump out the >> results of ALL my templates, but as far as I can tell it only prints out the >> template of the rule I called. >> >> I've debugged the resulting .java file and I can step into the results from >> each rule and see that a correct template is created. >> But I can't figure out how to "walk" all the templates and have them dump >> out my results. >> >> >From the reading I've done this morning I'm beginning to believe that >> the templates aren't expected to do this... >> >> Can someone clear this up for me? >> Point me in the direction of a template example that demonstrates this if it >> is suppose to work. >> >> Any help is appreciated. >> >> Morgan >> >> List: http://www.antlr.org/mailman/listinfo/antlr-interest >> Unsubscribe: >> http://www.antlr.org/mailman/options/antlr-interest/your-email-address >> >> >> >>
[il-antlr-interest: 34966] Re: [antlr-interest] Failure to ignore newline
Hi Bart -
Yes, it's a \n, and I thought I told the grammar to set '\n' to a hidden
channel. So, why is it not hidden?
~ Dave
On Fri, Nov 18, 2011 at 12:21 AM, Bart Kiers wrote:
> On Fri, Nov 18, 2011 at 7:55 AM, David Riddle wrote:
>
>> Hi -
>>
>> This should be a very simple thing - I'm attempting to have my grammar
>> hide
>> newline, carriage returns, etc. However, every concievable form of a
>> grammar that attempts to skip over these things or send them to the hidden
>> channel seems to fail for me. Here's a very basic example:
>>
>> grammar Test;
>>
>> prog: ID+;
>>
>> ID: 'a'..'z'+;
>>
>> WS: '\n'+ {$channel=HIDDEN;};
>>
>> // Input: a \n b
>> // Output: a n b
>>
>
> I'm guessing that's no 0xA (new line char) in your input, but a backslash
> followed by a 'n'.
>
> Regards,
>
> Bart.
>
>
--
David Riddle
Servoy Developer
(510) 854-6221
www.mcgilly.com
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34967] Re: [antlr-interest] Failure to ignore newline
On Fri, Nov 18, 2011 at 4:01 PM, David Riddle wrote: > Hi Bart - > > Yes, it's a \n, and I thought I told the grammar to set '\n' to a hidden > channel. So, why is it not hidden? > > Assuming you mean a new line char, then it _is_ being sent to the HIDDEN channel as Norman already mentioned. Regards, Bart. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34968] [antlr-interest] [C] reuse() bug here?
Hi Jim,
I think bug is here ... Just only have found this place during debug,
And need run away for few hours.
static pANTLR3_COMMON_TOKEN
newPoolToken(pANTLR3_TOKEN_FACTORY factory)
{
pANTLR3_COMMON_TOKEN token;
/* See if we need a new token pool before allocating a new
* one
*/
if (factory->nextToken >= ANTLR3_FACTORY_POOL_SIZE)
<< 1 !! If we have FEW pools then this is wrong.
{
/* We ran out of tokens in the current pool, so we need a new pool
*/
newPool(factory);
}
/* Assuming everything went well (we are trying for performance here so
doing minimal
* error checking. Then we can work out what the pointer is to the next
token.
*/
token = factory->pools[factory->thisPool] + factory->nextToken;
^^
// RZ: nextToken was 1024, we have
allocate above new pool
// and we should use its ZERO item
factory->nextToken++; // and we get 1025 ...
It seems to me ... We must nextToken counter drop to zero when we allocate
next pool.
And may be correct other places ...
--
Best regards,
Ruslan Zasukhin
VP Engineering and New Technology
Paradigma Software, Inc
Valentina - Joining Worlds of Information
http://www.paradigmasoft.com
[I feel the need: the need for speed]
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34969] Re: [antlr-interest] Failure to ignore newline
Afternoon David,
On 18/11/11 15:01, David Riddle wrote:
> Yes, it's a \n, and I thought I told the grammar to set '\n' to a hidden
> channel. So, why is it not hidden?
Think in C, where a newline character is represented in code as '\n'.
There are two physical characters between the quotes, but the result is
a single character 0x0a if you like - a unix new line.
>>> WS: '\n'+ {$channel=HIDDEN;};
WS is defining a single character of whitespace, the newline, not the
two characters backslash and n.
>>> // Input: a \n b
You have passed in a string of characters "a \n b" which is 6 characters
long. You need to pass in "ab" which will be 3 characters.
>>> // Output: a n b
When I ran it, I actually got a /n b as output using your input.
I think you are trying to get the input reader to convert '\n' to a
single newline, and it obviously doesn't. Just press the return key when
you want a new line.
HTH
Cheers,
Norm.
Disclaimer: I'm not a compiler writer nor do I play one on TV.
--
Norman Dunbar
Dunbar IT Consultants Ltd
Registered address:
Thorpe House
61 Richardshaw Lane
Pudsey
West Yorkshire
United Kingdom
LS28 7EL
Company Number: 05132767
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34973] Re: [antlr-interest] String concatenation expression rule
Hi Bart, thanks very for the quick reply.
I should have made it clear that concatenating is only legal if at least one of
the operands is a string literal - that is where I am having an issue. In other
terms the input
3 3
should cause a syntax error, while
3 "" 3
should not (and evaluates to the string "33")
Franck
From: Bart Kiers [via ANTLR]
To: franck102
Sent: Friday, November 18, 2011 1:11 PM
Subject: Re: String concatenation expression rule
On Fri, Nov 18, 2011 at 12:39 PM, franck102 <[hidden email]> wrote:
> I am writing a grammar for a fairly complex expression language, and in
> particular I need to support string concatenation which is performed simply
> by separating string literals with a space; and which automatically
> converts
> other expressions to a string if needed to concatenate:
> "a" "b" -> "ab"
> 2+3 "mm" -> "5mm"
>
> I suspect I could use predicates to write a rule like this:
>
> concatExpression
> : ( expression | STRING_LITERAL )+ { apply only if at least
> one of the elements is a string literal }?
>
> Is there a way to achieve this? The alternative formulations I can think of
> are pretty messy...
>
> As far as I understand it, you don't need any predicate. I see a
concat-expression has a lower precedence than addition, in which case this
could do the trick:
grammar T;
options {
output=AST;
}
tokens {
ROOT;
CONCAT;
}
parse
: (expression ';')* EOF -> ^(ROOT expression*)
;
expression
: (add -> add) (add+ -> ^(CONCAT add+))?
;
add
: atom (('+' | '-')^ atom)*
;
atom
: Number
| String
| '(' expression ')' -> expression
;
Number : '0'..'9'+ ('.' '0'..'9'+)?;
String : '"' ~'"'* '"';
Space : ' ' {skip();};
You can test it with the following class:
import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.*;
public class Main {
public static void main(String[] args) throws Exception {
String src = "42 - 2; 2 + 3 \"mm\"; \"a\" \"b\" 4-3-2 \"c\"; \"pi = \"
3.14159;";
TLexer lexer = new TLexer(new ANTLRStringStream(src));
TParser parser = new TParser(new CommonTokenStream(lexer));
CommonTree root = (CommonTree)parser.parse().getTree(); ;
System.out.println(new DOTTreeGenerator().toDOT(root));
}
}
Regards,
Bart.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
If you reply to this email, your message will be added to the discussion below:
http://antlr.1301665.n2.nabble.com/String-concatenation-expression-rule-tp7007921p7008016.html
To unsubscribe from String concatenation expression rule, click here.
NAML
--
View this message in context:
http://antlr.1301665.n2.nabble.com/String-concatenation-expression-rule-tp7007921p7008934.html
Sent from the ANTLR mailing list archive at Nabble.com.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34974] Re: [antlr-interest] String concatenation expression rule
Don't make the parser trap that error as a syntax error. It is a semantic
error and so you should parser any operand type, then reject the incorrect
types with code.
jim
> -Original Message-
> From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
> boun...@antlr.org] On Behalf Of franck102
> Sent: Friday, November 18, 2011 8:53 AM
> To: antlr-interest@antlr.org
> Subject: Re: [antlr-interest] String concatenation expression rule
>
> Hi Bart, thanks very for the quick reply.
>
> I should have made it clear that concatenating is only legal if at
> least one of the operands is a string literal - that is where I am
> having an issue. In other terms the input
>
> 3 3
> should cause a syntax error, while
> 3 "" 3
> should not (and evaluates to the string "33")
>
>
> Franck
>
>
>
>
> From: Bart Kiers [via ANTLR] node+s1301665n7008016...@n2.nabble.com>
> To: franck102
> Sent: Friday, November 18, 2011 1:11 PM
> Subject: Re: String concatenation expression rule
>
>
> On Fri, Nov 18, 2011 at 12:39 PM, franck102 <[hidden email]> wrote:
>
>
> > I am writing a grammar for a fairly complex expression language, and
> in
> > particular I need to support string concatenation which is performed
> simply
> > by separating string literals with a space; and which automatically
> > converts
> > other expressions to a string if needed to concatenate:
> > "a" "b" -> "ab"
> > 2+3 "mm" -> "5mm"
> >
> > I suspect I could use predicates to write a rule like this:
> >
> > concatExpression
> >:( expression | STRING_LITERAL )+ { apply only if at
> least
> > one of the elements is a string literal }?
> >
> > Is there a way to achieve this? The alternative formulations I can
> think of
> > are pretty messy...
> >
> > As far as I understand it, you don't need any predicate. I see a
> concat-expression has a lower precedence than addition, in which case
> this
> could do the trick:
>
> grammar T;
>
> options {
> output=AST;
> }
>
> tokens {
> ROOT;
> CONCAT;
> }
>
> parse
> : (expression ';')* EOF -> ^(ROOT expression*)
> ;
>
> expression
> : (add -> add) (add+ -> ^(CONCAT add+))?
> ;
>
> add
> : atom (('+' | '-')^ atom)*
> ;
>
> atom
> : Number
> | String
> | '(' expression ')' -> expression
> ;
>
> Number : '0'..'9'+ ('.' '0'..'9'+)?;
> String : '"' ~'"'* '"';
> Space : ' ' {skip();};
>
> You can test it with the following class:
>
> import org.antlr.runtime.*;
> import org.antlr.runtime.tree.*;
> import org.antlr.stringtemplate.*;
>
> public class Main {
> public static void main(String[] args) throws Exception {
> String src = "42 - 2; 2 + 3 \"mm\"; \"a\" \"b\" 4-3-2 \"c\"; \"pi =
> \"
> 3.14159;";
> TLexer lexer = new TLexer(new ANTLRStringStream(src));
> TParser parser = new TParser(new CommonTokenStream(lexer));
> CommonTree root = (CommonTree)parser.parse().getTree(); ;
> System.out.println(new DOTTreeGenerator().toDOT(root));
> }
> }
>
> Regards,
>
> Bart.
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address
>
>
>
>
> If you reply to this email, your message will be added to the
> discussion below:
> http://antlr.1301665.n2.nabble.com/String-concatenation-expression-
> rule-tp7007921p7008016.html
> To unsubscribe from String concatenation expression rule, click here.
> NAML
>
> --
> View this message in context:
> http://antlr.1301665.n2.nabble.com/String-concatenation-expression-
> rule-tp7007921p7008934.html
> Sent from the ANTLR mailing list archive at Nabble.com.
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34975] Re: [antlr-interest] String concatenation expression rule
I guess I will try that if I have to, although this is an existing language
(with a custom, unmaintainable parser) that is already very ambiguous, and I am
concerned that parsing any sequence of pretty much anything as a string
concatenation is going to make things much worse in that respect.
Franck
From: Jim Idle [via ANTLR]
To: franck102
Sent: Friday, November 18, 2011 6:47 PM
Subject: Re: String concatenation expression rule
Don't make the parser trap that error as a syntax error. It is a semantic
error and so you should parser any operand type, then reject the incorrect
types with code.
jim
> -Original Message-
> From: [hidden email] [mailto:antlr-interest-
> [hidden email]] On Behalf Of franck102
> Sent: Friday, November 18, 2011 8:53 AM
> To: [hidden email]
> Subject: Re: [antlr-interest] String concatenation expression rule
>
> Hi Bart, thanks very for the quick reply.
>
> I should have made it clear that concatenating is only legal if at
> least one of the operands is a string literal - that is where I am
> having an issue. In other terms the input
>
> 3 3
> should cause a syntax error, while
> 3 "" 3
> should not (and evaluates to the string "33")
>
>
> Franck
>
>
>
>
> From: Bart Kiers [via ANTLR] [hidden email]>
> To: franck102 <[hidden email]>
> Sent: Friday, November 18, 2011 1:11 PM
> Subject: Re: String concatenation expression rule
>
>
> On Fri, Nov 18, 2011 at 12:39 PM, franck102 <[hidden email]> wrote:
>
>
> > I am writing a grammar for a fairly complex expression language, and
> in
> > particular I need to support string concatenation which is performed
> simply
> > by separating string literals with a space; and which automatically
> > converts
> > other expressions to a string if needed to concatenate:
> > "a" "b" -> "ab"
> > 2+3 "mm" -> "5mm"
> >
> > I suspect I could use predicates to write a rule like this:
> >
> > concatExpression
> > : ( expression | STRING_LITERAL )+ { apply only if at
> least
> > one of the elements is a string literal }?
> >
> > Is there a way to achieve this? The alternative formulations I can
> think of
> > are pretty messy...
> >
> > As far as I understand it, you don't need any predicate. I see a
> concat-expression has a lower precedence than addition, in which case
> this
> could do the trick:
>
> grammar T;
>
> options {
> output=AST;
> }
>
> tokens {
> ROOT;
> CONCAT;
> }
>
> parse
> : (expression ';')* EOF -> ^(ROOT expression*)
> ;
>
> expression
> : (add -> add) (add+ -> ^(CONCAT add+))?
> ;
>
> add
> : atom (('+' | '-')^ atom)*
> ;
>
> atom
> : Number
> | String
> | '(' expression ')' -> expression
> ;
>
> Number : '0'..'9'+ ('.' '0'..'9'+)?;
> String : '"' ~'"'* '"';
> Space : ' ' {skip();};
>
> You can test it with the following class:
>
> import org.antlr.runtime.*;
> import org.antlr.runtime.tree.*;
> import org.antlr.stringtemplate.*;
>
> public class Main {
> public static void main(String[] args) throws Exception {
> String src = "42 - 2; 2 + 3 \"mm\"; \"a\" \"b\" 4-3-2 \"c\"; \"pi =
> \"
> 3.14159;";
> TLexer lexer = new TLexer(new ANTLRStringStream(src));
> TParser parser = new TParser(new CommonTokenStream(lexer));
> CommonTree root = (CommonTree)parser.parse().getTree(); ;
> System.out.println(new DOTTreeGenerator().toDOT(root));
> }
> }
>
> Regards,
>
> Bart.
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address
>
>
>
>
> If you reply to this email, your message will be added to the
> discussion below:
> http://antlr.1301665.n2.nabble.com/String-concatenation-expression-
> rule-tp7007921p7008016.html
> To unsubscribe from String concatenation expression rule, click here.
> NAML
>
> --
> View this message in context:
> http://antlr.1301665.n2.nabble.com/String-concatenation-expression-
> rule-tp7007921p7008934.html
> Sent from the ANTLR mailing list archive at Nabble.com.
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
If you reply to this email, your message will be added to the discussion below:
http://antlr.1301665.n2.nabble.com/String-concatenation-expression-rule-tp7007921p7009129.html
To unsubscribe from String concatenation expression rule, click here.
NAML
--
View this message in context:
http://antlr.1301665.n2.nabble.com/String-concatenation-expression-rule-tp7007921p7009337.html
Sent from the ANTLR mailing list archive at
[il-antlr-interest: 34977] Re: [antlr-interest] [C] reuse() bug here? -- IGNORE it ... Not here :)
On 11/18/11 5:12 PM, "Ruslan Zasukhin" wrote: -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34978] [antlr-interest] [C] reuse() - grow of RAM -- attempt #2.
Hi Jim,
So I have debug debug and have found that
=
1) Generated Parser contains ctx->adaptor, which contains one more
tokenFactory. And for this factory NEVER is called reset().
I have add call reset() for now in the method of generated parser.
This have made things better, but still I see at least two players...
So next player is:
=
2) This adaptor also has more deeply hidden factory of trees.
And it is called a lots for nilNode().
And this factory
typedefstruct ANTLR3_ARBORETUM_struct
Although is very similar to
typedefstruct ANTLR3_TOKEN_FACTORY_struct
In work with pools, it do NOT have reset() function.
So I think I will try add reset() method to this struct,
And call it from generated parser reset() method as following
static void
SqlParser_v3ParserReset (pSqlParser_v3Parser ctx)
{
RECOGNIZER->reset(RECOGNIZER);
// RZ added this to see if this fixes grow of RAM.
ADAPTOR->tokenFactory->reset( ADAPTOR->tokenFactory );
((pANTLR3_COMMON_TREE_ADAPTOR)(ADAPTOR->super))->arboretum->reset(
(ADAPTOR->super))->arboretum);
}
--
Best regards,
Ruslan Zasukhin
VP Engineering and New Technology
Paradigma Software, Inc
Valentina - Joining Worlds of Information
http://www.paradigmasoft.com
[I feel the need: the need for speed]
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
