[il-antlr-interest: 34943] Re: [antlr-interest] C# problem with the gettext()
The C# code uses C# conventions in many (hopefully most) places. input.LT(1).Text Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of karen Sent: Thursday, November 17, 2011 6:25 AM To: antlr-interest@antlr.org Subject: [antlr-interest] C# problem with the gettext() Hi, I am new at ANTLR and I am having a problem trying to use the C# code generated with the Kuka KRL Grammar... I am using antlrworks version 1.4.3 , Visual Studio 2010. While compiling the C# code, Visual Studio claims that it doesn't recognize the getText() that appears like this: input.LT(1).getText(). I have tried to use different Dotnet libraries but it is still not working. Can anyone please help me? -- View this message in context: http://antlr.1301665.n2.nabble.com/C-problem-with-the-gettext-tp7003942p7003 942.html Sent from the ANTLR mailing list archive at Nabble.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34791] Re: [antlr-interest] .g vs .g4
Hi Kirby, The extension comes into play when searching for imported grammars. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Kirby Bohling Sent: Monday, November 07, 2011 12:21 PM To: Terence Parr Cc: antlr-interest Interest Subject: Re: [antlr-interest] .g vs .g4 On Mon, Nov 7, 2011 at 12:01 PM, Terence Parr pa...@cs.usfca.edu wrote: yeah, maybe forcing tool to do .g4 is good otherwise why use it. .g4 means v4. .g means v3 (er…and v2). Ter Please don't do that. If I want to call that file foobar.g, or silly what does it matter? Have a declaration inside of the file if need be, or be one of the wonderful handful and just add it as part of the documentation at the top of the grammar? When it doesn't work, I promise, I'll get around to reading the docs at the top. Every time I have a tool which refuses to do my bidding it just ends up getting in my way. What if I want to pipe it in over stdin so there is no file name? Virtually every time I write or use a tool with tight constraints, later on I come across the counter example where it is a big work around to get where I want to be because a tool decided to dictate minor details. It really irritates me that unzip thinks the file _has_ to be named .zip, .z, or .Z. I _know_ it is a zip file, just unzip the thing, stop guessing at the file name. Should the community say it is a best practice? Absolutely, but I'm really happy that the C/C++ pre-processor doesn't force me to name the files foo.h, or foo.hpp to get them to work. At the very least, put in the flags/switches for expert mode, which will override any of the magic auto-detection, so when I tell it what to, it just does it. I have no idea in what context that'll be useful, but in my experience, there's always one. Kirby On Nov 7, 2011, at 9:59 AM, Jim Idle wrote: We started down this path with v3 and never really took it anywhere. I would be in favor of using .g4 only. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Terence Parr Sent: Sunday, November 06, 2011 9:56 AM To: Eric Cc: antlr-interest Interest Subject: Re: [antlr-interest] .g vs .g4 Hi Eric,I think we are going to allow .g as well. Rather than a property in the grammar file itself that identifies the version, I think we were going to use the file extension for this reason. all v3 grammars will be .g and people building v4 grammar should name them .g4 Ter On Nov 6, 2011, at 9:29 AM, Eric wrote: On Sun, Nov 6, 2011 at 12:22 PM, Terence Parr pa...@cs.usfca.edu wrote: Hi, Sam Harwell and I are talking about the file extension for new ANTLR 4. We think it makes good sense to use .g4 to distinguish the new grammars. They're mostly compatible except that there is no need for syntactic predicates and .g4 allows immediate left recursion. I think it will reduce confusion about what version of ANTLR is necessary to compile a grammar. Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr- interest/your-email-address Hi Ter, Will v4 require the file type to be .g4, or will it accept .g and .g4? My preference is for .g4 only. Also, will there be a property in the grammar file that identifies the minimum version or required version. I think it would help for obvious reasons such as when a user post a grammar needing help or an example grammar is given. Thanks, Eric List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34717] Re: [antlr-interest] delimiters directive does not work with C# version of StringTemplate
Hi Maik, Currently the delimiters syntax only works with group files (TemplateGroupFile, *.stg), not standalone template files (TemplateGroupDirectory, *.st). Based on the error message you listed, I'm guessing this is the problem? Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of pragmaik Sent: Friday, November 04, 2011 2:43 AM To: antlr-interest@antlr.org Subject: [antlr-interest] delimiters directive does not work with C# version of StringTemplate I'd like to generate some XML documents using StringTemplate and C#, so I've used delimiters $, $ to change the default delimiters. Unfortunately, it didn't work and I've got the following error: System.ArgumentException: Can't find template arg.st I get the same error when I set the default delimiters explicitly: delimiters , Cheers, Maik -- View this message in context: http://antlr.1301665.n2.nabble.com/delimiters-directive-does-not-work-with-C -version-of-StringTemplate-tp6961898p6961898.html Sent from the ANTLR mailing list archive at Nabble.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34741] Re: [antlr-interest] StringTemplate and Antrl3.runtime version mess on C# target
Hi Andy,
The Antlr3.Runtime.dll included in all of the packages is exactly the same.
It's used by ST3, ST4, the Tool, and the generated grammars for both the
CSharp2 and CSharp3 targets.
http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases
Sam
From: Voelkel, Andy [mailto:andy.voel...@plantronics.com]
Sent: Friday, November 04, 2011 1:51 PM
To: antlr-interest@antlr.org
Cc: Eric; Sam Harwell (sharw...@pixelminegames.com)
Subject: StringTemplate and Antrl3.runtime version mess on C# target
I've done some further investigation on this problem, and it seems that it
wasn't quite what I thought it was. As I suspected, it was connected to my
recent efforts to use Antlr generated StringTemplate code. I hadn't done
this previously because Antlr3.4 uses StringTemplate 4 for its own purposes,
but generates code that expects to use StringTemplate 3 if you use Antlr to
StringTemplate integration.
The problem is that I apparently need a different version of
Antlr3.Runtime.dll to run the C# code generated by Antlr3.exe than I need to
run StringTemplate 3, which that same generated code expects to use.
I got the version of Antlr3.Runtime.dll that StringTemplate3 expects from
the StringTemplate site, and tried to add that to my project along with the
version of Antlr.Runtime.dll that the generated C# code expects, but Visual
Studio doesn't like it.
What a mess! How do I resolve this? And I'm mystified, since it seems that
anyone who tries to use the latest version of the C# target and also wants
to have Antlr generate StringTemplate code will run into this problem.
- Andy
On Fri, Nov 4, 2011 at 1:13 PM, Voelkel, Andy andy.voel...@plantronics.com
wrote:
I thought I had things working based on the fix below. I ran Antlr, and
compiled a project based on its output without error. However, when I try to
run the project, I get:
An unhandled exception of type 'System.IO.FileLoadException' occurred in
Tree1.exe
Additional information: Could not load file or assembly 'Antlr3.Runtime,
Version=3.3.0.7239, Culture=neutral, PublicKeyToken=eb42632606e9261f' or one
of its dependencies. The located assembly's manifest definition does not
match the assembly reference. (Exception from HRESULT: 0x80131040)
Do I need to somehow rebuild/update the DLL so I can run it? Is it a .NET
security thing that refuses to use the modified .stg files? And somehow when
Antlr3.exe runs, it get around this?
- Andy
-Original Message-
From: Maik Schmidt [mailto:cont...@maik-schmidt.de]
Sent: Friday, November 04, 2011 12:21 AM
To: Voelkel, Andy
Subject: Re: [antlr-interest] This should be simple, but I can't find the
right syntax (or is it a bug?)
Hi Andy!
This is a know bug. Read
http://antlr.1301665.n2.nabble.com/Problems-with-Tree-Grammars-outputting-Te
mplates-in-C-td6724828.html
to find the solution.
Cheers,
Maik
On Fri, Nov 4, 2011 at 1:59 AM, Voelkel, Andy andy.voel...@plantronics.com
wrote:
Hi,
I'm trying to collect all the StringTemplates created by all instances of
one rule (borg) into a StringTemplate for a containing rule (prog). But I
can't seem to find the right syntax:
options {
output = template;
language = CSharp3;
tokenVocab=Tree1;
ASTLabelType = CommonTree;
}
public prog
: (b+=borg)+ - template(vals = {$b})
{ // prog
vals; separator = ,
}
;
The syntax I tried above is one many approaches I've tried - none seem to
work. Antlr3 runs without error but the code won't compile. I hope the
intent is clear, as it is a pretty simple example. I've looked back and
forth in Terence's book trying to find something like this, and in the
examples, but so far I've struck out. It seems like a very common thing to
try to do!
One strange thing is that when I look at the generated code, the list for
b (list_b) is declared as ListCommonTree. But Terence's books says is
should be ListStringTemplate when the output type is template. Later in
the generated code, when it tries to add to the list, it tries to create a
ListStringTemplate, which seems right, but causes a compiler error. It
almost seems like a bug, but I probably just don't understand.
Can anyone help me with this?
- Andy
CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files
or previous e-mail messages attached to it, may contain information that is
confidential and/or legally privileged. If you are not the intended
recipient, or a person responsible for delivering it to the intended
recipient, please DO NOT disclose the contents to another person, store or
copy the information in any medium, or use any of the information contained
in or attached to this transmission for any purpose. If you have received
this transmission in error, please immediately notify the sender by reply
email or at priv...@plantronics.com, and destroy the original transmission
and its attachments without reading or saving in any manner.
For further information
[il-antlr-interest: 34743] Re: [antlr-interest] StringTemplate and Antrl3.runtime version mess on C# target
I went ahead and edited the description of the Bootstrap release. Can you
check it and let me know if it makes things more clear?
Thanks,
Sam
From: Voelkel, Andy [mailto:andy.voel...@plantronics.com]
Sent: Friday, November 04, 2011 3:01 PM
To: Sam Harwell; antlr-interest@antlr.org
Cc: 'Eric'
Subject: RE: StringTemplate and Antrl3.runtime version mess on C# target
I see. I originally got the bootstrap release, which does not include ST3.
Then when I went to get ST3 after I discovered I needed it for Antlr
generated StringTemplate code, I got it from the StringTemplate website.
Hence the incompatibility.
Would you mind if I edited the Wiki Page for the C# target to make all of
this a little more clear to the uninitiated?
- Andy
From: Sam Harwell [mailto:sharw...@pixelminegames.com]
Sent: Friday, November 04, 2011 12:14 PM
To: Voelkel, Andy; antlr-interest@antlr.org
Cc: 'Eric'
Subject: RE: StringTemplate and Antrl3.runtime version mess on C# target
Hi Andy,
The Antlr3.Runtime.dll included in all of the packages is exactly the same.
It's used by ST3, ST4, the Tool, and the generated grammars for both the
CSharp2 and CSharp3 targets.
http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases
Sam
From: Voelkel, Andy [mailto:andy.voel...@plantronics.com]
Sent: Friday, November 04, 2011 1:51 PM
To: antlr-interest@antlr.org
Cc: Eric; Sam Harwell (sharw...@pixelminegames.com)
Subject: StringTemplate and Antrl3.runtime version mess on C# target
I've done some further investigation on this problem, and it seems that it
wasn't quite what I thought it was. As I suspected, it was connected to my
recent efforts to use Antlr generated StringTemplate code. I hadn't done
this previously because Antlr3.4 uses StringTemplate 4 for its own purposes,
but generates code that expects to use StringTemplate 3 if you use Antlr to
StringTemplate integration.
The problem is that I apparently need a different version of
Antlr3.Runtime.dll to run the C# code generated by Antlr3.exe than I need to
run StringTemplate 3, which that same generated code expects to use.
I got the version of Antlr3.Runtime.dll that StringTemplate3 expects from
the StringTemplate site, and tried to add that to my project along with the
version of Antlr.Runtime.dll that the generated C# code expects, but Visual
Studio doesn't like it.
What a mess! How do I resolve this? And I'm mystified, since it seems that
anyone who tries to use the latest version of the C# target and also wants
to have Antlr generate StringTemplate code will run into this problem.
- Andy
On Fri, Nov 4, 2011 at 1:13 PM, Voelkel, Andy andy.voel...@plantronics.com
wrote:
I thought I had things working based on the fix below. I ran Antlr, and
compiled a project based on its output without error. However, when I try to
run the project, I get:
An unhandled exception of type 'System.IO.FileLoadException' occurred in
Tree1.exe
Additional information: Could not load file or assembly 'Antlr3.Runtime,
Version=3.3.0.7239, Culture=neutral, PublicKeyToken=eb42632606e9261f' or one
of its dependencies. The located assembly's manifest definition does not
match the assembly reference. (Exception from HRESULT: 0x80131040)
Do I need to somehow rebuild/update the DLL so I can run it? Is it a .NET
security thing that refuses to use the modified .stg files? And somehow when
Antlr3.exe runs, it get around this?
- Andy
-Original Message-
From: Maik Schmidt [mailto:cont...@maik-schmidt.de]
Sent: Friday, November 04, 2011 12:21 AM
To: Voelkel, Andy
Subject: Re: [antlr-interest] This should be simple, but I can't find the
right syntax (or is it a bug?)
Hi Andy!
This is a know bug. Read
http://antlr.1301665.n2.nabble.com/Problems-with-Tree-Grammars-outputting-Te
mplates-in-C-td6724828.html
to find the solution.
Cheers,
Maik
On Fri, Nov 4, 2011 at 1:59 AM, Voelkel, Andy andy.voel...@plantronics.com
wrote:
Hi,
I'm trying to collect all the StringTemplates created by all instances of
one rule (borg) into a StringTemplate for a containing rule (prog). But I
can't seem to find the right syntax:
options {
output = template;
language = CSharp3;
tokenVocab=Tree1;
ASTLabelType = CommonTree;
}
public prog
: (b+=borg)+ - template(vals = {$b})
{ // prog
vals; separator = ,
}
;
The syntax I tried above is one many approaches I've tried - none seem to
work. Antlr3 runs without error but the code won't compile. I hope the
intent is clear, as it is a pretty simple example. I've looked back and
forth in Terence's book trying to find something like this, and in the
examples, but so far I've struck out. It seems like a very common thing to
try to do!
One strange thing is that when I look at the generated code, the list for
b (list_b) is declared as ListCommonTree. But Terence's books says is
should be ListStringTemplate when the output type is template. Later in
the generated code, when
[il-antlr-interest: 34666] Re: [antlr-interest] working examples for StringTemplate4 and C#? (I'm stuck again)
Do you have a stack trace or any other information about where the exception occurs? In Visual Studio, you can get a stack trace even if you don't have the source code by going to DebugExceptions..., checking the box next to UriFormatException, and then looking at the Call Stack window after the exception is thrown. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Voelkel, Andy Sent: Sunday, October 30, 2011 5:05 PM To: 'ANTLR interest' Subject: [antlr-interest] working examples for StringTemplate4 and C#? (I'm stuck again) Hi, I'm getting stuck trying to start using template group files. I try this to open my .stg file: TemplateGroup Tg = new TemplateGroupFile(RTX.stg, '', ''); .. and I get a UriFormatException, The format of the URI could not be determined. Are there any working examples of using ST4 with C# that anyone could point me to? I'll gladly update the ST4 Wiki (which seems to be out of date) as I get simple examples working. Or would I be better off just sticking with ST3? - Andy CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it, may contain information that is confidential and/or legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, please DO NOT disclose the contents to another person, store or copy the information in any medium, or use any of the information contained in or attached to this transmission for any purpose. If you have received this transmission in error, please immediately notify the sender by reply email or at priv...@plantronics.com, and destroy the original transmission and its attachments without reading or saving in any manner. For further information about Plantronics - the Company, its products, brands, partners, please visit our website www.plantronics.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34635] Re: [antlr-interest] Are token aliases possible?
In the Java version, the rewrite syntax is the definitive way to specify the
type of a tree node.
In the newer versions of the C# port, you can specify tree node types as a
token option, so you could do this and be equivalent to your rewrite rule.
Note that I added a ^ as well; your first example did not include it but
that's what makes a particular node the root of the created tree. Since the
AST operators (^ and !) perform better in a number of ways than the rewrite
operators, I added this feature to allow me to use AST operators in a number
of places where I previously used rewrite syntax for the sole purpose of
changing a node type.
class_declaration
: CLASSCLASS_DECL^ type_or_generic class_base?
type_parameter_constraints_clauses? class_body ';'!?
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Christian
Sent: Friday, October 28, 2011 6:46 AM
To: antlr-interest@antlr.org
Subject: Re: [antlr-interest] Are token aliases possible?
Yes, I know that one can use rewrite rules. Thank you anyway. In some cases
however, I do not want to repeat the rules following the CLASS token and
write an arrow etc. In such cases, it is shorter to just replace the CLASS
by CLASS_DECL and add a few operators instead of a rewrite rule.
As an example, I want to transform the following rule:
class_declaration
: CLASS type_or_generic class_base?
type_parameter_constraints_clauses? class_body ';'?
into
class_declaration
: CLASS_DECL type_or_generic class_base?
type_parameter_constraints_clauses? class_body ';'!?
versus
class_declaration
: CLASS type_or_generic class_base?
type_parameter_constraints_clauses? class_body ';'?
- ^(CLASS_DECL type_or_generic class_base?
type_parameter_constraints_clauses? class_body)
;
I think everyone, especially me, would prefer the first solution.
Am 28.10.2011 13:34, schrieb Sam Barnett-Cormack:
I don't know for sure, buut my immediate thought is that it isn't
likely to be possible (as tokens are generated without reference to
parse context, and must be unambiguous except where code manually
deals with the ambiguity). You could however create it as a dummy
token name, and rewrite to it in the parser, such that the AST
generated has a 'virtual' CLASS_DECL token generated from the 'real'
CLASS token. Pretty sure there are examples of this in the wiki.
On 28/10/2011 12:27, Christian wrote:
CLASS is only the token name. However, CLASS_DECL represents the
semantics of what was parsed. It allows to be more consequent when
analyzing the generated AST by names like CLASS_DECL, METHOD_DECL,
FIELD_DECL etc.
Example:
switch (node.getType()) {
case CsRewriteRulesParser.NAMESPACE_DECL:
// TODO
break;
case CsRewriteRulesParser.METHOD_DECL:
// TODO
break;
case CsRewriteRulesParser.CLASS_DECL:
// TODO
break;
}
It would be just another name for the integer constant CLASS.
Am 28.10.2011 13:19, schrieb Sam Barnett-Cormack:
I think the immediate answer is, why?
On 28/10/2011 09:02, Christian wrote:
Hi community,
if there is a real token, say
CLASS : 'class';
can a define an alias, say
CLASS_DECL = CLASS
in some way?
Regards,
Christian
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-addr
ess
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-addres
s
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34649] Re: [antlr-interest] tool version mismatch for C# target??
Hi Andy, I just checked the Wiki and the downloads, and see the inconsistency. ANTLR.exe uses StringTemplate for code generation, but now only depends on ST4. I ended up removing ST3 from the Bootstrap download as a result. You can download ST3 as a separate download from the wiki: http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Voelkel, Andy Sent: Friday, October 28, 2011 12:01 PM To: ANTLR interest Subject: [antlr-interest] tool version mismatch for C# target?? Hi, I'm starting to use the built in StringTemplate generation feature of ANTLR (see version below). It generates the following code near the beginning of the output file: using Antlr3.ST; using Antlr3.ST.Language; [System.CodeDom.Compiler.GeneratedCode(ANTLR, 3.4.1.9004)] But the version of StringTemplate I got with this same release is Antlr4.StringTemplate, not Antlr3.ST. How do I fix this up? - Andy CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it, may contain information that is confidential and/or legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, please DO NOT disclose the contents to another person, store or copy the information in any medium, or use any of the information contained in or attached to this transmission for any purpose. If you have received this transmission in error, please immediately notify the sender by reply email or at priv...@plantronics.com, and destroy the original transmission and its attachments without reading or saving in any manner. For further information about Plantronics - the Company, its products, brands, partners, please visit our website www.plantronics.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34652] Re: [antlr-interest] tool version mismatch for C# target??
No, as far as I know Antlr 3 will always produce code for ST3. ANTLR 4 will produce code for ST4 by default. Sam -Original Message- From: Voelkel, Andy [mailto:andy.voel...@plantronics.com] Sent: Friday, October 28, 2011 1:11 PM To: Sam Harwell; 'ANTLR interest' Subject: RE: [antlr-interest] tool version mismatch for C# target?? Hi Sam, Is there a version or Antlr3.exe that produces code which uses Antlr4.StringTemplate? That would the ideal, I would think. - Andy -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: Friday, October 28, 2011 10:52 AM To: Voelkel, Andy; 'ANTLR interest' Subject: RE: [antlr-interest] tool version mismatch for C# target?? Hi Andy, I just checked the Wiki and the downloads, and see the inconsistency. ANTLR.exe uses StringTemplate for code generation, but now only depends on ST4. I ended up removing ST3 from the Bootstrap download as a result. You can download ST3 as a separate download from the wiki: http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Voelkel, Andy Sent: Friday, October 28, 2011 12:01 PM To: ANTLR interest Subject: [antlr-interest] tool version mismatch for C# target?? Hi, I'm starting to use the built in StringTemplate generation feature of ANTLR (see version below). It generates the following code near the beginning of the output file: using Antlr3.ST; using Antlr3.ST.Language; [System.CodeDom.Compiler.GeneratedCode(ANTLR, 3.4.1.9004)] But the version of StringTemplate I got with this same release is Antlr4.StringTemplate, not Antlr3.ST. How do I fix this up? - Andy CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it, may contain information that is confidential and/or legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, please DO NOT disclose the contents to another person, store or copy the information in any medium, or use any of the information contained in or attached to this transmission for any purpose. If you have received this transmission in error, please immediately notify the sender by reply email or at priv...@plantronics.com, and destroy the original transmission and its attachments without reading or saving in any manner. For further information about Plantronics - the Company, its products, brands, partners, please visit our website www.plantronics.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it, may contain information that is confidential and/or legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, please DO NOT disclose the contents to another person, store or copy the information in any medium, or use any of the information contained in or attached to this transmission for any purpose. If you have received this transmission in error, please immediately notify the sender by reply email or at priv...@plantronics.com, and destroy the original transmission and its attachments without reading or saving in any manner. For further information about Plantronics - the Company, its products, brands, partners, please visit our website www.plantronics.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34654] Re: [antlr-interest] tool version mismatch for C# target??
That's why I said will ;) Still not sure when it will be released. I know Dr. Parr's been working on it for quite a while, but I believe it's still a few months away from a first beta release. I'm already excited about it though - looks like it will really shake things up (in a good way)! Sam -Original Message- From: Voelkel, Andy [mailto:andy.voel...@plantronics.com] Sent: Friday, October 28, 2011 2:11 PM To: Sam Harwell; 'ANTLR interest' Subject: RE: [antlr-interest] tool version mismatch for C# target?? I didn't know there was an ANTLR 4. The website advertises 3.4 as the latest version. - Andy -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: Friday, October 28, 2011 11:45 AM To: Voelkel, Andy; 'ANTLR interest' Subject: RE: [antlr-interest] tool version mismatch for C# target?? No, as far as I know Antlr 3 will always produce code for ST3. ANTLR 4 will produce code for ST4 by default. Sam -Original Message- From: Voelkel, Andy [mailto:andy.voel...@plantronics.com] Sent: Friday, October 28, 2011 1:11 PM To: Sam Harwell; 'ANTLR interest' Subject: RE: [antlr-interest] tool version mismatch for C# target?? Hi Sam, Is there a version or Antlr3.exe that produces code which uses Antlr4.StringTemplate? That would the ideal, I would think. - Andy -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: Friday, October 28, 2011 10:52 AM To: Voelkel, Andy; 'ANTLR interest' Subject: RE: [antlr-interest] tool version mismatch for C# target?? Hi Andy, I just checked the Wiki and the downloads, and see the inconsistency. ANTLR.exe uses StringTemplate for code generation, but now only depends on ST4. I ended up removing ST3 from the Bootstrap download as a result. You can download ST3 as a separate download from the wiki: http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Voelkel, Andy Sent: Friday, October 28, 2011 12:01 PM To: ANTLR interest Subject: [antlr-interest] tool version mismatch for C# target?? Hi, I'm starting to use the built in StringTemplate generation feature of ANTLR (see version below). It generates the following code near the beginning of the output file: using Antlr3.ST; using Antlr3.ST.Language; [System.CodeDom.Compiler.GeneratedCode(ANTLR, 3.4.1.9004)] But the version of StringTemplate I got with this same release is Antlr4.StringTemplate, not Antlr3.ST. How do I fix this up? - Andy CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it, may contain information that is confidential and/or legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, please DO NOT disclose the contents to another person, store or copy the information in any medium, or use any of the information contained in or attached to this transmission for any purpose. If you have received this transmission in error, please immediately notify the sender by reply email or at priv...@plantronics.com, and destroy the original transmission and its attachments without reading or saving in any manner. For further information about Plantronics - the Company, its products, brands, partners, please visit our website www.plantronics.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it, may contain information that is confidential and/or legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, please DO NOT disclose the contents to another person, store or copy the information in any medium, or use any of the information contained in or attached to this transmission for any purpose. If you have received this transmission in error, please immediately notify the sender by reply email or at priv...@plantronics.com, and destroy the original transmission and its attachments without reading or saving in any manner. For further information about Plantronics - the Company, its products, brands, partners, please visit our website www.plantronics.com. CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it, may contain information that is confidential and/or legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, please DO NOT disclose the contents to another person, store or copy the information in any medium, or use any of the information
[il-antlr-interest: 34584] Re: [antlr-interest] Exceptions on C#-Target
Hello,
I just ran your test, and it didn't throw any externally visible exceptions.
Internally I would expect a very large number of exceptions because your lexer
has no rules that match any of your input except comments, whitespace, the word
CREATE, and the semicolon at the end.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Anonym
Sent: Monday, August 29, 2011 6:02 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] Exceptions on C#-Target
i´m working with antlrworks 1.4.2
i have the following, extremly simple grammar (combined lexer and parser):
*
public sqlfile :
commands*
EOF
;
commands:
STARTWORD (options { greedy=false; } : .*) SEM
;
STARTWORD
: ('SELECT'|'INSERT'|'UPDATE'|'DELETE')
;
SEM : ';' ;
COMMENT : '/*' ( options { greedy=false; } : .*) '*/'
{
$channel = 2;
}
;
SHARP
: '#' ( options { greedy=false; } : .*) NL
{
$channel = 2;
}
;
SINGLELINECOMMENT
: '--' ( options { greedy=false; } : .*) NL
{
$channel = 2;
}
;
WS : (' '|'\t')+ { $channel = 5; } ;
NL : ('\r' '\n'? | '\n') { $channel = 5; } ;
*
i´ve debugged it in antlrworks and works fine (as expected). i used this
test-string:
*--
-- SQL to create the guestbook table
--
CREATE TABLE `guestbook` (
`id` int(11) NOT NULL auto_increment,
`TimeStamp` int(11) default '0',
`Email` varchar(60) default NULL,
`Name` varchar(100) NOT NULL default '',
`Website` varchar(60) default NULL,
`Title` varchar(255) default NULL,
`Message` text,
`Verified` smallint(1) NOT NULL default '0',
`Code` varchar(40) NOT NULL default '',
UNIQUE KEY `id` (`id`)
) TYPE=MyISAM AUTO_INCREMENT=6 ;
*
then i build against csharp3-target and used the parser in my
c#-application.
while executing the parser.sqlfile()-Method 317 Exceptions
(MissmatchToken and NoViableAlt) are thrown!
now i wonder why, because my grammar is REALLY simple and should match,
like it in the java-target did (no error in antlrworks-console).
In the C#-Output the exceptions are thrown at the line
STARTWORD1=(IToken)Match(input,STARTWORD,Follow._STARTWORD_in_commands135);
in commands()-Rule
where is my mistake?
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34504] Re: [antlr-interest] More problems with ANTRL C#3 target and tools
For the first problem, you probably have your grammar and helper files in
different namespaces. It looks like you're using a combined grammar, which
means you'll want this in the grammar:
@lexer::namespace{Namespace.Goes.Here}
@parser::namespace{Namespace.Goes.Here}
Then you'll use a matching namespace in your partial file.
For the second problem, just declare the rule public as mentioned in section
5.2 of the documentation:
http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases
Thanks,
Sam
From: Voelkel, Andy [mailto:andy.voel...@plantronics.com]
Sent: Friday, October 21, 2011 2:10 PM
To: antlr-interest@antlr.org
Cc: Sam Harwell (sharw...@pixelminegames.com)
Subject: More problems with ANTRL C#3 target and tools
Hi,
I got over a first round of problems and got a few trivial example working.
I then moved on to trying to convert on of the examples from The Definitive
Guide to ANTLR from Java-compatible to C# compatible:
http://media.pragprog.com/titles/tpantlr/code/tour/eval/Expr.g
It is a simple calculator example, before he introduces ASTs.
I have run into two problems. The first seems to be related to Sam's visual
studio integration tools, and particularly to the use of partial classes.
This seems in some cases to cause this error:
K:\Plantronics\Development
Sandbox\ANTLR\AntlrTest\AntlrTest3\Program.cs(13,30): error CS1729:
'AntlrTest3.Combined1Lexer' does not contain a constructor that takes 1
arguments
One of the partial class definitions for Combined1Lexer does not indicate
that Combined1Lexer inherits from Lexer and that presumably causes the
constructor not to be found. If I remove the file containing the partial
class definition from the project, that problem goes away. The funny thing
is that this problem seems to happen on some projects and not others, which
is weird. Another weird thing is that explicitly changing the partial class
definition to inherit from Lexer causes a different problem:
K:\Plantronics\Development
Sandbox\ANTLR\AntlrTest\AntlrTest3\Combined1.g3.lexer.cs(5,17): error
CS0534: 'AntlrTest3.Combined1Lexer' does not implement inherited abstract
member 'Antlr.Runtime.Lexer.mTokens()'
The second problem is that the example code in the book contains a call to
one of the rules, in my code this looks like this:
Combined1Parser parser = new Combined1Parser(tokens);
// Begin parsing at rule prog
parser.prog();
But the generated code for CombinedParser defines prog() as being a private
method, meaning that I can't call it. Can someone explain this?
Just when it looks like I'm gaining traction, I run into more weird stuff.
-Andy
P.S. I've run this example in Antlrworks with the Java-compatible version,
and everything works fine, so these problems all seem to be related to the
C# target.
_
CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files
or previous e-mail messages attached to it, may contain information that is
confidential and/or legally privileged. If you are not the intended
recipient, or a person responsible for delivering it to the intended
recipient, please DO NOT disclose the contents to another person, store or
copy the information in any medium, or use any of the information contained
in or attached to this transmission for any purpose. If you have received
this transmission in error, please immediately notify the sender by reply
email or at priv...@plantronics.com, and destroy the original transmission
and its attachments without reading or saving in any manner.
For further information about Plantronics - the Company, its products,
brands, partners, please visit our website www.plantronics.com.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34456] Re: [antlr-interest] How viable is the Csharp3 target?
Hi Andy Maik, I agree that the C# target documentation is lacking in many ways. I try to keep enough documentation at the following page to get you started on the right path: http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases I've been using the CSharp3 target in a commercial product for 3 years now. At this point, most of the remaining the issues I have with it are basic/technical limitations of ANTLR 3 that will hopefully be resolved in later versions. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of pragmaik Sent: Wednesday, October 19, 2011 9:36 AM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] How viable is the Csharp3 target? Hi Andy! I am using ANTLR's C# target for a couple of weeks now to create a domain specific language that is a subset of the C programming language. My experience is similar to yours, but I can assure you that the C# target works and that it is actively maintained. It differs from the Java target in a few places, but it works. The biggest problem is finding documentation. I've both of Terence Parr's books on my desk and the folks on this mailing list are very friendly and competent. Cheers, Maik -- View this message in context: http://antlr.1301665.n2.nabble.com/How-viable-is-the-Csharp3-target-tp690889 0p6908949.html Sent from the ANTLR mailing list archive at Nabble.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34227] Re: [antlr-interest] Re : incremental
Hi J. Rampon, I have an application in which individual files frequently change, and reparsing the entire set of 4000 files would take a significant amount of time. I handle this situation by keeping the parsed information in a loosely bound cache, so my incremental updates are simply reparsing a single file. ANTLR does not have any inherent ability to perform incremental parsing, and in my experience would not be particularly useful. Carefully written grammars can certainly analyze in excess of 100KB of source code/second (mine in C# are closer to 1MB/sec). Do you have a particular application where this approach would fail, but incremental parsing within the file would succeed? Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Rampon Jerome Sent: Friday, September 30, 2011 12:33 PM To: antlr-interest@antlr.org Subject: [antlr-interest] Re : incremental Hi, Compilers/parsers are often used over files slightly modified (error correction, whatever minor changes, ) Efficient applications may desire to implement incremental abilities (to speed-up runtime). Those incremental abilities can be integrated at different levels (application, parser, ...). Of course, the sooner you can collect data/changes, the best you incremental mechanism will be. I did not find any topic related about in ANTLR documentation for such any support. If anything exists, could you please let me know as part of my interest ? Thanks J.Rampon List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34205] Re: [antlr-interest] Csharp target, Recover(input, e)
I've never seen this occur before. Can you send me a project with a grammar and input that exhibits this? Thanks, Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Mari Matinlassi Sent: Wednesday, September 28, 2011 10:37 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Csharp target, Recover(input, e) Hi, I am doing a grammar for C sharp target. I have following problem, I need help with… When parser input contains unexpected character, let´s say ‘*’ or ‘’ that is not defined in any lexer/parser rule or token, happens the following: 1. RecognitionException (subtype MismatchedTokenException)will be thrown from the rule in question. This OK. 2. Error reported is ok. Something like “mismatched input [@20,56:55='',18,8:2] expecting ENDOFCLASS” 3. Recover method will be called. This takes forever(or at least 10 seconds), then another exception will be thrown. Either “OutOfMemoryException” or “ArgumentOutOfRangeException” Shouldn´t Recover(input, e) method recover from this kinds of errors? What am I doing wrong? Thanks for help, Mari List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34052] Re: [antlr-interest] Need quick ANTLR/ST3 hack
I believe if(rest(args)) will evaluate to true if args has more than one
item.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Bill Andersen
Sent: Wednesday, September 14, 2011 1:26 PM
To: antlr-interest
Subject: [antlr-interest] Need quick ANTLR/ST3 hack
Folks
I want to write a template containing a conditional expansion for a
multi-valued attribute that does one thing if the attribute is singleton and
another if there are more than 1 element
For example, in an ANTLR grammar with ST output
rule
: FOO (bars+=bar)+ - foo(args={$bars})
;
where the template looks like
foo(args) ::=
if(???)
something
else
something else
endif
I want some expression for ??? to test for args being singleton.
Is this possible in ST3? Else I imagine I could write:
rule
: FOO (bars+=bar)+ - foo(args={$bars},singleton={$bars.size() == 1})
;
and
foo(args,singleton) ::=
if(singleton)
something with args
else
something else with args
endif
... but I would really rather not.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34006] Re: [antlr-interest] ANTLRWorks 2 (for ANTLR v4)
This is easy for a parser. Just add the following to your grammar.
@rulecatch{}
Aside from answering the single question In the input recognized by this
grammar? (yes/no), this is a rather useless thing to do. There are many,
many other ways to approach error handling that are more appropriate for any
other task.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Wujek Srujek
Sent: Monday, September 12, 2011 4:18 AM
To: antlr-interest Interest
Subject: Re: [antlr-interest] ANTLRWorks 2 (for ANTLR v4)
I am new to ANTLR3 and haven't yet got a lot of experience with it. The one
thing that is bugging me, though, is how hard it was to make sure ANTLR
doesn't perform any recovery and fails on first error with checked
exceptions. And it involved a nasty trick with sneaky throwing... I would
love to see it made easier.
wujek
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 34008] Re: [antlr-interest] ANTLRWorks 2 (for ANTLR v4)
In these cases, you'll want to override the Recover method in your generated
grammar and throw a new exception (derived from RuntimeException, not
RecognitionException) containing better information about the cause of the
error. In .NET I throw an OperationCanceledException. This should work for
all types of grammars.
This uses an unchecked but appropriate exception for the situation. Allowing
user-specified checked exceptions in a mix of generated and library code is
unnecessary and impractical.
Sam
From: Wujek Srujek [mailto:wujek.sru...@googlemail.com]
Sent: Monday, September 12, 2011 10:40 AM
To: Sam Harwell
Subject: Re: [antlr-interest] ANTLRWorks 2 (for ANTLR v4)
Hi.
I think the @rulecatch has to be specified per rule, which makes it at least
cumbersome for pretty much all grammars. Please correct me if I am wrong.
How about the lexer?
Do you mean failing on first error is useless? I think it is not - [removed
example before sending to public list]. I don't want any recovery here, with
some artificial tokens inserted for me - I want the whole [.] to fail. As I
was not the only one who searched for an answer (the ANTLR site has a
dedicated site for it) I guess it is not so uncommon.
I agree that recovery is nice in some cases, but not in all of them.
Thank you for your answer.
wujek
On Mon, Sep 12, 2011 at 5:26 PM, Sam Harwell sharw...@pixelminegames.com
wrote:
This is easy for a parser. Just add the following to your grammar.
@rulecatch{}
Aside from answering the single question In the input recognized by this
grammar? (yes/no), this is a rather useless thing to do. There are many,
many other ways to approach error handling that are more appropriate for any
other task.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Wujek Srujek
Sent: Monday, September 12, 2011 4:18 AM
To: antlr-interest Interest
Subject: Re: [antlr-interest] ANTLRWorks 2 (for ANTLR v4)
I am new to ANTLR3 and haven't yet got a lot of experience with it. The one
thing that is bugging me, though, is how hard it was to make sure ANTLR
doesn't perform any recovery and fails on first error with checked
exceptions. And it involved a nasty trick with sneaky throwing... I would
love to see it made easier.
wujek
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33960] Re: [antlr-interest] ANTLRWorks 2 (for ANTLR v4)
With a carefully designed algorithm, autocomplete actually works quite well in an ANTLR grammar. I went over portions of the algorithm with Ter in the past, but it's outside the scope of what he's working on for ANTLR 4. A partial implementation will be included in the next release of my Visual Studio integration (clean and working, but missing a few features I've designed on paper). The new parsing algorithms I'm using do a great job of handling incomplete syntax, but use a very different type of parser than ANTLR generates. They also allow local analysis of a fragment of text within a file with a constant-time complexity WRT the size of the file, so it performs as well while editing a 1MB grammar as it does with a 1KB grammar. I'm hoping to create a modified version of the ANTLR tool that can produce these parsers from an ANTLR grammar. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Jim Idle Sent: Thursday, September 08, 2011 10:41 AM To: Antlr-Interest Antlr.Org Subject: Re: [antlr-interest] ANTLRWorks 2 (for ANTLR v4) That's an ANTLR request (which has already been considered), not a Works request. However, the new IDE should use a smarter parser that can handle incomplete syntax better - as in, it is coded to recognize such things where it is possible. Note that autocomplete does not really work for ANTLR grammars as almost all tokens can be a possibility in most places. Jim List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33964] Re: [antlr-interest] For getCharPositionInLine(), is there a way to make tab count for 8 chars instead of 1?
I actually asked Dr. Parr recently about why the property is called CharPositionInLine instead of just Column. The name was actually picked to make it clear that tab and space are each treated as 1 character. When talking about parsing, the term Column also always refers to the character position within a line. The notion that the width of a tab is not equal to the width of a space is a UI concept independent of the parsing problem, and widely varies as you move from IDEs to word processors to typesetting systems. Whether it's called CharPositionInLine or Column, the parser will provide consistent information about character positions and you'll need to independently implement the width of your tab characters as it applies to your particular target environment. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Dejas Ninethousand Sent: Thursday, September 08, 2011 11:59 AM To: antlr-interest@antlr.org Subject: [antlr-interest] For getCharPositionInLine(), is there a way to make tab count for 8 chars instead of 1? I want to ensure that line and cols reported by my parser line up with what the user would see in a program like Textpad. In Textpad if you move the cursor across a tab character the column counter and cursor advance by 8 and not 1 even though only one char in the stream has been traversed. Can I configure some setting in ANTRL such that tab chars are treated the same way for column reporting? Thanks. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33936] Re: [antlr-interest] Problems with Visual Studio 2010 and C#
Hi Amelie, You can find the latest version of the C# targets along with instructions for setting up your project in Visual Studio at the following location: http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases Thanks, Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Mueller, Amelie Sent: Wednesday, September 07, 2011 8:13 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Problems with Visual Studio 2010 and C# Hello, I'm currently working at a Project in cooperation with Thermo Fisher Scientific. We have chosen to take ANTLR as a tool . Right in the beginning a Problem appeared, when I try to run just a Example. I´m working with Visual Studio 2010 and use a C# Windows Form Application where I want to implement a Parser. I`m using the new Version 1.4.3 of Antlr and reference is the Antrl3.Runtime.dll. The Problem is that in the generated Code (in CSharp2 or CSharp3 ) is a Method that is not known from none Library public AstParserRuleReturnScopeCommonTree, CommonToken compileUnit() I don't know if it is just a simple problem because I missed something or if the problem appear because of the new version of the CSharp Libraries. Fact is that I don't found this method somewhere(google, antlr homepage). Thanks for your help, best regards Amelie Mueller Software Tests Development Thermo Scientific Cytomat Products Scientific Instruments - Laboratory Automation ThermoFisher Scientific Robert-Bosch-Str. 1 D-63505 Langenselbold, Germany List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33899] Re: [antlr-interest] Using Tree Interface not plain Object
The new interfaces aren't required across the board, but would be used for the vast majority of situations. By specifying the ASTLabelType=x; and TreeAdaptorType=y; options, you could override the entire system. This restructuring primarily affects the arguments and return values from various methods in the Tree and TreeAdaptor interfaces. Also, the new interfaces are MUCH simpler than they used to be. Methods which alter the tree are all moved to and unified in the TreeAdaptor, and many of the utility-type methods are moved to a new module instead of requiring every tree implementation to implement them. The previous interfaces were large, confusing (do you call tree.addChild or treeAdaptor.addChild?), and severely restricted my ability to create generic trees and a general purpose tree visualizer (similar to STViz but for parse results). Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Oliver Zeigermann Sent: Sunday, September 04, 2011 2:49 PM To: Terence Parr Cc: Antlr List Subject: Re: [antlr-interest] Using Tree Interface not plain Object What would be the benefit of introducing and then requiring those interfaces? Might make sense when you want to do automatic tree traversal, but maybe not when you want to create any kind of object where you might not even have a (single) parent or a flat list of children. Would it be possible to have both? Tree interfaces for homogeneous, normalized trees that can be processed further using ANTLR and whatever you want to generate if you do not want further ANTLR processing? - Oliver 2011/9/4 Terence Parr pa...@cs.usfca.edu: Hi. Currently v3 requires nothing of your tree nodes; all you have to do is give me an adapter that says how to create them and navigate them and manipulate them. Users can set up ASTLabelType as an option so that $x in actions has the user's type instead of plain object. Sam Harwell and I were talking today about restructuring all of the tree interfaces. We're breaking it up into a hierarchy with Tree interface at the top. Tree just says you can get a payload, some children, and parent. ParseTree and AST split off as sub interfaces. We were thinking that it's pretty silly to use simple object when we could require the minimal Tree interface so at least code could display or navigate them. The reason I went with plain object originally was so people could use any existing object they have, even if they only have binary for it. This seems like a fairly extreme example and, in the worst case, somebody could wrap those objects so they were suitable for use with ANTLR. More than likely, somebody in that situation would simply use actions to construct the trees they want. Most trees not built for antlr expect everything in the constructor, whereas antlr has to build up the trees piecemeal as it parses. So, does anybody object to ANTLR assuming Tree in the parser if you are building trees and then AST in the tree parsers? Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33763] Re: [antlr-interest] How to use StringTemplate 4 with C#
In ANTLR 3, the output=template option is only able to produce code for
ST3. You'll have to remove that option and write all your own actions if you
want to target ST4.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Maik Schmidt
Sent: Thursday, August 25, 2011 6:28 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] How to use StringTemplate 4 with C#
Hi!
I am writing a tree grammar with the following header
tree grammar Translator;
options {
language=CSharp3;
tokenVocab=Test;
ASTLabelType=CommonTree;
output = template;
}
My problem is that the generated code contains statements such as using
Antlr3.ST. I have added a reference to the Antl3 StringTemplate DLL but I
am wondering how I can use version 4.
Cheers,
Maik
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33606] Re: [antlr-interest] CSharp3 code generation suggestions
1. I’ll make this change for all references to the Antlr namespace (using
statements and base classes). To change the base class of a grammar, you can
use “@lexer::superClass{global::Antlr.Runtime.Lexer}” (non-combined grammars
may omit the lexer::, and any type name may be specified as the superClass).
2. You can use @lexer::modifier{internal} to control access to the classes.
Use @lexer::ctorModifier{private} if you’ve declared your own constructor(s)
and need to prevent external code from using the generated ones. Section 5.1
of the C# 3 documentation shows this new feature (added for the 3.4
release).
3. I use Code Contracts with rewriting extensively in one of my other (side)
projects. For now it’s a love/hate relationship, but the real reason I
haven’t annotated the existing code is there are some big questions about
the compatibility of the CC library with the BSD license used for the
runtime.
Sam
From: Ranco Marcus [mailto:ranco.mar...@epirion.nl]
Sent: Friday, August 12, 2011 3:59 AM
To: antlr-interest@antlr.org
Cc: Sam Harwell [sharw...@pixelminegames.com]
Subject: CSharp3 code generation suggestions
Hi Sam,
I have a few rather small suggestions for the code generated by the CSharp3
target.
1. We used to have a namespace .Antlr in the same project as where our
grammars are located. The compiler fails when it tries to resolve the
generated grammar's base classes (Antlr.Runtime.___) and the included
namespaces (using Antlr.___ ). Maybe you could make this explicit by
prefixing those reference with global::.
2. The generated grammars are public by default. If the visibility modifier
would be omitted from the partial class declaration, the grammar would be
internal by default (something I usually prefer). In case somebody wants the
grammar to be public, he/she only needs to add another partial class
definition with a public modifier.
3. We are currently in the process of annotating our code with code
contracts (see
http://msdn.microsoft.com/en-us/devlabs/dd491992.aspx, separate assembly in
.NET 3.5, included in the BCL in 4.0). The static checker (freely available
for VS Premium or above) is able to check after compilation where contracts
might be violated. Very cool stuff and maybe something to consider for the
Antlr runtime.
These are only suggestions, as none of the above are show-stoppers.
Best regards,
Ranco
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33587] Re: [antlr-interest] Using ANTLR in Visual C# 2010 Express?
The Visual Studio Express Editions do not support any extensions other than simple code templates. You *should* be able to use the ANTLR extension with the free Visual Studio 2010 Shell (Integrated Mode), but I haven't tried this. Here are links to the installer and the SP1 update. Visual Studio 2010 Shell (Integrated Mode): http://www.microsoft.com/download/en/details.aspx?id=115 Visual Studio 2010 SP1 Update: http://www.microsoft.com/download/en/details.aspx?id=23691 Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of leodido Sent: Thursday, August 11, 2011 2:41 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Using ANTLR in Visual C# 2010 Express? Hi all, I'm wondering if it is possible to use ANTLR with the free version Visual C# 2010 Express? I'm really confused about now because I found two different guides: http://www.antlr.org/wiki/pages/viewpage.action?pageId=557075# and http://www.tunnelvisionlabs.com/downloads/antlr/ANTLRCSharpDocumentation-900 4.xps I tried following both and none of them works. I have an issue while following the ANTLRCSharpDocumentation-9004.xps guide because apparently you can't install all types of extension in the version of Visual Studio I'm using. It says This extension is not installable on any currently installed products. Also I can't seem to find the line in my .sln file that needs to be modified. I haven't even done anything in my code and it won't compile giving me errors. (ie. The type or namespace name 'GrammarRuleAttribute' could not be found (are you missing a using directive or an assembly reference?) OR The type or namespace name 'String' could not be found (are you missing a using directive or an assembly reference?)) I generated my cs files using antlrworks-1.4.3.jar and target CSharp2 and CSharp3 (Which target should I use?) and using the C Grammar file available on the ANTLR page (just added the line language = 'CSharp2'; and output=AST;) Could a generous soul come to my rescue? I'd also be willing to write a small document to clearly describe how to use ANTLR with C# if I ever succeed into doing so :p Thanks in advance. -- View this message in context: http://antlr.1301665.n2.nabble.com/Using-ANTLR-in-Visual-C-2010-Express-tp66 75388p6675388.html Sent from the ANTLR mailing list archive at Nabble.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33599] Re: [antlr-interest] ANTLR build process performance improvement
ANTLR is designed so with the lexer and parser as independent units. Your
lexer should be able to run to completion (the entire document) before the
parser is even created.
There are ways around it, but I intentionally avoid getting into situations
where I have to think about what they might be.
Sam
From: chris king [mailto:kingce...@gmail.com]
Sent: Thursday, August 11, 2011 2:59 PM
To: Sam Harwell; antlr-interest@antlr.org
Subject: Re: ANTLR build process performance improvement
Hey Sam, thanks! I'm using it now and it's faster. Love the VS tool-chain!
I'd be dead in the water without it. And haven't bumped into any bugs with
the new version. So cool!
I've bummed into a slightly different problem and no matter how I arrange my
grammar (included) I can't seem to work around it. I think it might be a bug
in the SpecialStateTransition logic...
Below is what I'm trying to parse along with the trace using your enter-exit
partial methods (wonderful addition BTW) for my C# preprocessor interleaved
with when the tokens get pulled from the stream. Also interleaved are my
calls to toggle SkipSection which is trying to keep track of when code is
#ifdefed out. So what we see is that [#if] is pulled and then [false] and
[\r\n]. At that point in the parse I'm at the end of the pre-processor line
and so I look up the pp_conditional stack to see if I'm in a #ifdefed out
section of code. In this case I am so I set SkipSection to True. That
enables (via semantic predicate) my lexer rule PP_SKIPPED_CHARACTERS=45
which should suck up any code that is not a pragma statement (doesn't start
with #). That's all well and good and so the next thing that get tokenized
is [#pragma warning disable] which is good.
Now at this point I expect that no tokens should get pulled until I reach
the pp_pragma production. I expect this because I figure ANLTER should be
able to predict where it needs to go without pulling any more tokens --
after all the only thing that can follow a [#pragma warning disable] token
is a list of integers. The actual behavior is that in my
pp_condition_section production ANTLR pulls [10] as a PP_SKIPPED_CHARACTERS
instead of an INTEGER because SkipSection is set to True. If things had gone
as expected and [10] had been pulled in the pp_pragma production. If that
had happened then SkipSection would have been set to False and [10] would be
pulled as an INTEGER.
The code that's actually pulling the [10] is DFA.Predict in the
SpecialStateTransition loop when trying to predict where to go for the
pp_conditional_section production:
pp_conditional_section
: { !SkipSection }? = input_section
| { SkipSection }? = pp_skipped_section
;
I'm guessing that this production is special because it's got those gated
semantic predicates and that's why DFA.Predict enters into the
SpecialStateTransition logic. What I don't understand is why it would need
to pull any more tokens to know where to go next. Do you think that's a bug
that it's pulling tokens in this case?
After writing this e-mail it occurred to me that I might manually try to do
the prediction. I did this by putting a break point in the
pp_conditional_section rule at the dfa.Predict line. But instead of asking
the DFA to do the prediction I just set-ip to the case I wanted (e.g. {...}?
= pp_skipped_section). Then hit F5. And it works! I included the trace of
that run below. Given that I really do think that the SpecialStateTransition
logic (or there abouts) is being to aggressive about pulling tokens... what
do you think?
Thanks,
Chris
CSharpAst.Parse(#if false\r\n#pragma warning disable
10\r\n/*foo*/\r\n#endif);
Enter start 1
[@-1,0:2='#if',38,1:0]
Enter input_section 2
Enter input_section_part 3
Enter pp_directive 6
Enter pp_conditional 8
Enter pp_if_section 9
[@-1,4:8='false',4,1:4]
Enter pp_expression 17
Enter pp_or_expression 18
Enter pp_and_expression 19
Enter pp_equality_expression 20
Enter pp_unary_expression 21
Enter pp_primary_expression 22
[@-1,9:10='\\r\\n',29,1:9]
Leave pp_primary_expression 22
Leave pp_unary_expression 21
Leave pp_equality_expression 20
Leave pp_and_expression 19
Leave pp_or_expression 18
Leave pp_expression 17
Enter pp_conditional_block 12
Enter pp_new_line 31
SkipSection = True
[@-1,11:33='#pragma warning disable',42,2:0]
Leave pp_new_line 31
Enter pp_conditional_section 13
[@-1,34:36=' 10',45,2:23]
[@-1,37:38='\\r\\n',29,2:26]
Enter pp_skipped_section 14
Enter pp_skipped_section_part 15
Enter pp_directive 6
Enter pp_leaf_directive 7
Enter pp_pragma 29
SkipSection = False
Enter pp_warning_list 30
Here is the trace when I make the prediction by hand:
Enter start 1
[@-1,0:2='#if',38,1:0]
Enter input_section 2
Enter input_section_part
[il-antlr-interest: 33575] Re: [antlr-interest] Access to line numbers
Both the Tree and Token interfaces have a method getLine(). Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of David Smith Sent: Wednesday, August 10, 2011 9:02 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Access to line numbers It seems that the parse tree generated by the parser has a data field 'line' that seems to correspond to the source line when parsing. How do i access that to include the line number in the tree generated by my grammar? DMS David M. Smith http://www.cc.gatech.edu/fac/David.Smith Georgia Institute of Technology, College of Computing Sent from my ASR-33 Teletype List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33579] Re: [antlr-interest] To Sam Harwell
Hi Chris,
I fixed this for the next release, but I don't have a build ready yet. If
you'd like to make the change locally so you can keep working, just do the
following:
1. Add the following at the beginning of
SemanticContext.And(SemanticContext, SemanticContext):
if (a is FalsePredicate || b is FalsePredicate)
return FalsePredicate.Instance;
2. Add the following at the beginning of SemanticContext.Or(SemanticContext,
SemanticContext):
if (a is TruePredicate || b is TruePredicate)
return TruePredicate.Instance;
Sam
From: chris king [mailto:kingce...@gmail.com]
Sent: Tuesday, August 09, 2011 11:47 PM
To: Sam Harwell
Cc: antlr-interest@antlr.org
Subject: Re: [antlr-interest] To Sam Harwell
Oops. Let's try this again...
On Tue, Aug 9, 2011 at 9:41 PM, Sam Harwell sharw...@pixelminegames.com
wrote:
Hi Chris,
You attached CSharp.cs instead of CSharp.g3, so I currently can't reproduce
this issue.
I'm glad you caught this. It definitely looks like a bug in some work I
recently did on the predicate analyzer.
I'll have to get this taken care of tomorrow since it's late here. :)
Thanks,
Sam
From: chris king [mailto:kingce...@gmail.com]
Sent: Tuesday, August 09, 2011 9:56 PM
To: Sam Harwell; antlr-interest@antlr.org
Subject: Re: [antlr-interest] To Sam Harwell
Sam, hey. Sorry to shower you with e-mails today. :) I got your new version
setup up and building but now one of my grammar file is causing a stack
overflow. I'm not sure if this is a problem with the CSharp port or a bug in
the Java ANTLR logic. Anyway, the attached grammer file of mine worked with
v3.3.1.7705 but with v3.4.1.9004 is causing a stack overflow in antlr3.exe.
To repro include the attached file in a .csproj as an Antlr3 include per
your instructions. Then uncomment either one of these rules and save to
execute Antlr3.
pp_conditional_section
: //{ !SkipSection }? = input_section
| //{ SkipSection }? = pp_skipped_section_part*
;
Here's how I'm include the file in the project:
Antlr3 Include=CSharp.g
GeneratorMSBuild:Compile/Generator
/Antlr3
Here's the top of the function signature that I'm recursing through:
// Factor so (a || b) == (result || a || b)
public static SemanticContext FactorOr(ref SemanticContext a, ref
SemanticContext b)
{
HashSetSemanticContext opsA = new
HashSetSemanticContext(GetOrOperands(a));
HashSetSemanticContext opsB = new
HashSetSemanticContext(GetOrOperands(b));
Here's a stack trace of the recursion I'm seeing. The line numbers should
match the source in v3.4.1.9004. I used that source to build the Antlr3.exe
that's having the stack overflow.
Antlr3.EXE!Antlr3.Analysis.SemanticContext.And(a, b) Line 681
C#
Antlr3.EXE!Antlr3.Analysis.SemanticContext.And(a, b) Line 681
C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 809 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 730 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 823 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 740 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 740 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 823 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3
[il-antlr-interest: 33557] Re: [antlr-interest] To Sam Harwell
Hi Chris, I got rid of the use of StackT and the call to ElementAt() quite a while back. Instead I use a ListStackT which derives from ListT and adds Push and Pop methods. I recommend updating to ANTLR 3.4.1 to correct this issue. Sam From: chris king [mailto:kingce...@gmail.com] Sent: Tuesday, August 09, 2011 6:29 PM To: Sam Harwell; antlr-interest@antlr.org Subject: Re: [antlr-interest] To Sam Harwell Sam, hey, so I might have bumped into a code gen bug. I'm trying to access a variable skipSection on the parent production of pp_conditional. It looks like the generated code is doing some computation involving the stack count when really it should just pass my index without any calculation. For example, below, shouldn't the highlighted code simply be my index? Or am I missing something? Looks like the highlighted code is assuming TOS is Count - 1 (like it would be with a stack implemented with a ListT) but I don't think that's the case. The stack is a System.Collections.Generic.StackT which maintains that TOS is always index 0. $pp_conditional[1]::skipSection The code above is transformed to: System.Linq.Enumerable.ElementAt(pp_conditional_stack, pp_conditional_stack.Count-1-1).skipSection; Also, you may want to ensure that all user expressions are enclosed in parens. As I was trying to work around this issue I used count -2 which got translated to stack.Count-count -2 -1. I had to add the parens to get it to be stack.Count-(count -2) -1. Thanks, Chris On Thu, Aug 4, 2011 at 6:00 PM, chris king kingce...@gmail.com wrote: Ok. Thanks for looking into it. On Thu, Aug 4, 2011 at 5:56 PM, Sam Harwell sharw...@pixelminegames.com wrote: I did, and I can repro the issue but I haven't resolved it yet. Sam From: chris king [mailto:kingce...@gmail.com] Sent: Thursday, August 04, 2011 6:48 PM To: Sam Harwell Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] To Sam Harwell Awesome! I'll give it try. Did you see my follow up email that the parse string is 5 characters (there is a trailing space)? /**/ And again, love the tool chain. I only really started to get traction on my project after I installed it. It's working great. Thanks, Chris On Thu, Aug 4, 2011 at 4:34 PM, Sam Harwell sharw...@pixelminegames.com wrote: Hi Chris, In build 3.4.1.9004 that I released today, I switched all the projects to using $(ProjectDir) with relative paths. http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of chris king Sent: Monday, July 25, 2011 6:39 PM To: antlr-interest@antlr.org Subject: [antlr-interest] To Sam Harwell Sam, hey, hope this finds you. Very small suggestion follows :). In Antlr3.StringTemplate.csproj could you use AntlrBuildTaskPath$(MSBuildProjectDirectory)\..\bin\Bootstrap/AntlrBuildT askPath to reference the bootstrap directory? Originally it used the solutionDir and that prevented me from including a subset of the projects in my project (so I could simply reference them and have all the debugging, pdb, source, etc just work). Thanks, Chris List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33559] Re: [antlr-interest] To Sam Harwell
Hi Chris,
You attached CSharp.cs instead of CSharp.g3, so I currently can't reproduce
this issue.
I'm glad you caught this. It definitely looks like a bug in some work I
recently did on the predicate analyzer.
I'll have to get this taken care of tomorrow since it's late here. :)
Thanks,
Sam
From: chris king [mailto:kingce...@gmail.com]
Sent: Tuesday, August 09, 2011 9:56 PM
To: Sam Harwell; antlr-interest@antlr.org
Subject: Re: [antlr-interest] To Sam Harwell
Sam, hey. Sorry to shower you with e-mails today. :) I got your new version
setup up and building but now one of my grammar file is causing a stack
overflow. I'm not sure if this is a problem with the CSharp port or a bug in
the Java ANTLR logic. Anyway, the attached grammer file of mine worked with
v3.3.1.7705 but with v3.4.1.9004 is causing a stack overflow in antlr3.exe.
To repro include the attached file in a .csproj as an Antlr3 include per
your instructions. Then uncomment either one of these rules and save to
execute Antlr3.
pp_conditional_section
: //{ !SkipSection }? = input_section
| //{ SkipSection }? = pp_skipped_section_part*
;
Here's how I'm include the file in the project:
Antlr3 Include=CSharp.g
GeneratorMSBuild:Compile/Generator
/Antlr3
Here's the top of the function signature that I'm recursing through:
// Factor so (a || b) == (result || a || b)
public static SemanticContext FactorOr(ref SemanticContext a, ref
SemanticContext b)
{
HashSetSemanticContext opsA = new
HashSetSemanticContext(GetOrOperands(a));
HashSetSemanticContext opsB = new
HashSetSemanticContext(GetOrOperands(b));
Here's a stack trace of the recursion I'm seeing. The line numbers should
match the source in v3.4.1.9004. I used that source to build the Antlr3.exe
that's having the stack overflow.
Antlr3.EXE!Antlr3.Analysis.SemanticContext.And(a, b) Line 681
C#
Antlr3.EXE!Antlr3.Analysis.SemanticContext.And(a, b) Line 681
C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 809 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 730 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 823 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 740 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 740 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 823 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 730 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 813 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 730 C#
Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt,
context, semanticContext, d, collectPredicates) Line 760 C#
Antlr3.EXE!Antlr3
[il-antlr-interest: 33513] [antlr-interest] Optimizations for CSharp3 target with large grammars and output=AST
Hello everyone, I'm working on codegen optimization for grammars with output=AST. Medium to large grammars should see a significant improvement in the way local variables are handled in rules, *especially* when rewrite operators are used. Since this is currently an experimental optimization, I'd prefer the initial testers be advanced users familiar with both the creation/usage of grammars and the C# code generated by ANTLR. If you have a grammar which would particularly benefit from this and want to help test this version, let me know and I'll send you information about the preview release. Here is a summary of some the optimizations included: . Label aliasing where possible for both tokens and rule return values. . Removal of Rewrite*Stream variables which are not actually referenced by a rule's rewrite blocks. . Removal of local variables for tokens and rules which are not referenced in an action and either aren't in the rewrite rule or have a bang(!) operator. Thanks, Sam List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33483] Re: [antlr-interest] Have I found an Antlr CSharp3 lexer bug if...
Hi Chris,
I'm using the released version 3.4.0 of the ANTLR CSharp3 target. I
copy/pasted the grammar below (aside from renaming it to Preprocessor) and
it passed the following unit test.
[TestMethod]
public void TestEmptyComment()
{
string inputText = /**/;
var input = new ANTLRStringStream(inputText);
var lexer = new PreprocessorLexer(input);
var tokenStream = new CommonTokenStream(lexer);
tokenStream.Fill();
ListIToken tokens = tokenStream.GetTokens();
Assert.AreEqual(2, tokens.Count);
Assert.AreEqual(PreprocessorLexer.DELIMITED_COMMENT, tokens[0].Type);
Assert.AreEqual(inputText, tokens[0].Text);
Assert.AreEqual(PreprocessorLexer.EOF, tokens[1].Type);
}
Sam
From: chris king [mailto:kingce...@gmail.com]
Sent: Thursday, August 04, 2011 3:48 AM
To: Sam Harwell; antlr-interest@antlr.org
Subject: Re: Have I found an Antlr CSharp3 lexer bug if...
Sam, while trying build my pre-processor with a mixed parser/lexer I ran
across what I think might be a bug. I reduced the repro below. I expected
the program below to accept /**/ but instead fails because the lexer
prediction enters PP_SKIPPED_CHARACTERS. That rule has a gated semantic
predicate which is always false. I expected a lexer rule with a gated
semantic predicate which is always false to never be matched. If I comment
out the PP_SKIPPED_CHARACTERS rule then it does match /**/ . So the
inclusion of that rule is cause the problem. Let me know if you think this
is a bug and if you can repro.
Thanks,
Chris
grammar Bug;
options {
language=CSharp3;
output=AST;
}
public start
: DELIMITED_COMMENT !EOF
;
PP_SKIPPED_CHARACTERS
: { false }? = ~(F_NEW_LINE_CHARACTER | F_PP_POUND_SIGN)
F_INPUT_CHARACTER*
;
DELIMITED_COMMENT
: { true }? = '/*' .* '*/'
;
WHITESPACE
: F_WHITESPACE {skip();}
;
fragment F_WHITESPACE
: (' ' | '\t' | '\v' | '\f')+
;
fragment F_NEW_LINE_CHARACTER
: '\r'
| '\n'
;
fragment F_PP_POUND_SIGN
: '#'
;
fragment F_INPUT_CHARACTER
: ~F_NEW_LINE_CHARACTER
;
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33497] Re: [antlr-interest] To Sam Harwell
Hi Chris, In build 3.4.1.9004 that I released today, I switched all the projects to using $(ProjectDir) with relative paths. http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of chris king Sent: Monday, July 25, 2011 6:39 PM To: antlr-interest@antlr.org Subject: [antlr-interest] To Sam Harwell Sam, hey, hope this finds you. Very small suggestion follows :). In Antlr3.StringTemplate.csproj could you use AntlrBuildTaskPath$(MSBuildProjectDirectory)\..\bin\Bootstrap/AntlrBuildT askPath to reference the bootstrap directory? Originally it used the solutionDir and that prevented me from including a subset of the projects in my project (so I could simply reference them and have all the debugging, pdb, source, etc just work). Thanks, Chris List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33439] [antlr-interest] C# updates for ANTLR 3.4, StringTemplate 4.0.4
Hello everyone, I just finished updating the C# ports of ANTLR and StringTemplate for the recent releases of ANTLR 3.4 and StringTemplate 4.0.4. This is a significant update for the C# port, and includes the following in addition to the updates found in the corresponding Java versions. . The CSharp2 target has been updated to use the same runtime assembly as the CSharp3 target (Antlr3.Runtime.dll). . Tree grammars with filter=true now automatically generate the required overridden implementations of TreeFilter.Topdown and/or TreeFilter.Bottomup. . Improved support for delegate grammars in the build process (set their Build Action to AntlrAbstractGrammar and Custom Tool to MSBuild:Compile). . Improved Tool performance. The documentation and downloads can be found at the following page: http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases Also, if you end up using (and liking) my Visual Studio 2010 extensions, I appreciate the reviews on the Visual Studio Gallery. :) Thank you, Sam Harwell List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33429] Re: [antlr-interest] Fwd: Any HTML editors for rich editing of documents conforming to an ANTLR grammar?
Hi Colin, I wasn't trying to say the DSL is a bad idea. I only meant that creating a full-featured editor is extremely challenging, and tends to not work as expected in the end (any combination of slow, inaccurate, and/or inconsistent). If you focus on providing syntax highlighting along with a background parser that can display syntax errors, it shouldn't be difficult to create a very usable tool. The level of difficulty also depends heavily on the complexity of the language and what your accuracy demands are for the auto-complete process. Currently I'm experimenting with Java, with my goals being extremely fast IDE assist operations (under 25ms latency) and a high degree of accuracy and consistency even in documents with multiple syntax errors. For the task of identifying the current context, I'm using a new parsing algorithm that I designed specifically for this task and might not apply to any other application. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Colin Yates Sent: Friday, July 29, 2011 11:04 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Fwd: Any HTML editors for rich editing of documents conforming to an ANTLR grammar? (oops - clicked reply instead of reply-all) -- Forwarded message -- From: Colin Yates colin.ya...@gmail.com Date: 29 July 2011 16:34 Subject: Re: [antlr-interest] Any HTML editors for rich editing of documents conforming to an ANTLR grammar? To: Sam Harwell sharw...@pixelminegames.com Hi Sam, Thanks for the info. I had seen both of those, but they are too heavyweight for my needs - the editor must be a HTML5 construct. Shipping any desktop tool is a no-no as this is a web-application. Regarding You probably shouldn't try to create a new solution in this area. I've been working on IDEs for the past several years, and I can tell the task is nowhere near as straightforward as it seems. Not sure what to think of that - there are quite a few assumptions in there :). I shall take it as a friendly warning of here be dragons as I am sure that is the intent. Unfortunately I don't see any other option. We have decided that a DSL is the right way forward for allowing clients to describe their reality to our tool (previously it was through a chinese-whispers process of spreadsheets into spreadsheets which we parsed and populated templates in the persistent store - *that* is nowhere near as straightforward as it seems :)), and we have decided that ANTLR is the right tool to allow us to build the DSL. I am surprised that there isn't more traffic in this area - I would have thought my use-case is very common. If it is, then how do others allow clients to build non-trivial documents conforming to a grammar defined in ANTLR? If I do have to do this from scratch then I expect I shall: - define an abstraction for the things the editor cares about (current errors, ability to identify context based on cursor position, safe suggestions for example) - provide an implementation that is hard coded for one document in the first instance - dive into the internals in the second instance to see how much I can grab from the ANTLR internals of the generated tokeniser and parser in the second instance Creating a HTML5 construct to appear to work is pretty trivial I expect (and I cannot believe it hasn't already been done). I see the 2nd and 3rd points above being the can-of-worms, particularly the ability to figure out the relevant part of the grammar that matches the place in the document the user is currently editing. With respect, I see it as non-trivial, but not un-doable. It would make a nice open-source project as well Thanks for the help - I don't mean to be dismissive - keep the ideas coming :) Col On 29 July 2011 16:16, Sam Harwell sharw...@pixelminegames.com wrote: Hi Colin, In very general terms, the task at hand is you'd like to create an IDE for a language described by an ANTLR grammar. I haven't used either of these products, but the closest groups I know of to producing a product like you describe below are Xtext: http://www.eclipse.org/Xtext/ and Actipro SyntaxEditor: http://www.actiprosoftware.com/products/dotnet/wpf/syntaxeditor/default.aspx You probably shouldn't try to create a new solution in this area. I've been working on IDEs for the past several years, and I can tell the task is nowhere near as straightforward as it seems. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Colin Yates Sent: Friday, July 29, 2011 5:32 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Any HTML editors for rich editing of documents conforming to an ANTLR grammar? Hi all, Most of the products we develop use documents which are parsed by ANTLR to describe the real world to the application. These are none-trivial and need to be internally consistent
[il-antlr-interest: 33399] Re: [antlr-interest] Have I found an Antlr CSharp3 lexer bug if...
Hi Chris,
Lookahead prediction occurs before predicates are evaluated. If fixed
lookahead uniquely determines the alternative with a semantic predicate,
the predicate will not be evaluated as part of the decision process. I'm
guessing (but not 100% sure) if you use a gated semantic predicate, then it
will not be entering the rule:
PP_SKIPPED_CHARACTERS
: {false}? = ( ~(F_NEW_LINE_CHARACTER | '#') F_INPUT_CHARACTER*
F_NEW_LINE )*
;
Also, a word of warning: this lexer rule can match a zero-length character
span, which could result in an infinite loop. You should always ensure that
every path through any lexer rule that's not marked fragment will consume
at least 1 character. There's also a bug with certain exceptions in the
lexer that can cause infinite loops - this has been resolved for release 3.4
but I haven't released it yet.
Sam
From: chris king [mailto:kingce...@gmail.com]
Sent: Thursday, July 28, 2011 4:19 PM
To: antlr-interest@antlr.org; Sam Harwell
Subject: Have I found an Antlr CSharp3 lexer bug if...
Have I found an Antlr lexer CSharp3 bug if I can alter program execution
(exception instead of no exception) by introducing a lexer production with a
predicate that is always false? For example
PP_SKIPPED_CHARACTERS
: { false }? ( ~(F_NEW_LINE_CHARACTER | '#') F_INPUT_CHARACTER* F_NEW_LINE
)*
;
I would think that such a production should always be ignored because it's
predicate is always false and therefore would never alter program execution.
Yet I'm seeing a change in the execution of my program. I'm seeing it enter
this function and throw a FailedPredicateException. I wouldn't have expected
that this function should ever even have been executed because the predicate
is always false.
[GrammarRule(PP_SKIPPED_CHARACTERS)]
private void mPP_SKIPPED_CHARACTERS()
{
EnterRule_PP_SKIPPED_CHARACTERS();
EnterRule(PP_SKIPPED_CHARACTERS, 31);
TraceIn(PP_SKIPPED_CHARACTERS, 31);
try
{
int _type = PP_SKIPPED_CHARACTERS;
int _channel = DefaultTokenChannel;
// CSharp\\CSharpPreProcessor.g:197:3: ({...}? (~ (
F_NEW_LINE_CHARACTER | F_POUND_SIGN ) ( F_INPUT_CHARACTER )
DebugEnterAlt(1);
// CSharp\\CSharpPreProcessor.g:197:5: {...}? (~ (
F_NEW_LINE_CHARACTER | F_POUND_SIGN ) ( F_INPUT_CHARACTER )
{
DebugLocation(197, 5);
if (!(( false )))
{
throw new FailedPredicateException(input,
PP_SKIPPED_CHARACTERS, False() );
}
Sam, I'm on an all CSharp stack v3.3.1.7705. I'm using your VS plugin (which
is wonderful) and build integration to generate the lexer/parser (also
wonderful) and then running on top of your CSharp port of the runtime. If
you think this is a bug and you'd like to have a look at the repro please
let me know. The project is open source up on CodePlex.
Thanks,
Chris
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33404] Re: [antlr-interest] Interest in a Go lang port
Hi Parsiad, I believe he's referring to creating a new ANTLR target/runtime for Go (option language=Go), as opposed to using ANTLR to compile Go files into executable code. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Parsiad Azimzadeh Sent: Thursday, July 28, 2011 7:51 PM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] Interest in a Go lang port Would anyone be interested in collaborating on a port of ANTLR to the Go lanaguage http://golang.org/? Interesting. What are your motivations for porting Go to ANTLR? I believe the Go compiler is open source. -- Parsiad Azimzadeh http://sfu.ca/~paa4 List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33407] Re: [antlr-interest] Have I found an Antlr CSharp3 lexer bug if...
IncludedCodeState {
public readonly bool FoundElseDirective;
private readonly bool? _isIncluded;
public IncludedCodeState(bool? isIncluded, bool foundElseDirective) {
_isIncluded = isIncluded;
FoundElseDirective = foundElseDirective;
}
public bool IsIncluded { get { return _isIncluded ?? false; } }
public IncludedCodeState ElseState {
get {
if (_isIncluded == null)
return new IncludedCodeState(_isIncluded, true);
return new IncludedCodeState(!_isIncluded, true);
}
}
}
}
Sam
From: chris king [mailto:kingce...@gmail.com]
Sent: Thursday, July 28, 2011 7:05 PM
To: Sam Harwell
Cc: antlr-interest@antlr.org
Subject: Re: Have I found an Antlr CSharp3 lexer bug if...
Sam, thanks so much for taking the time to look at that. If I could, let me
try and explain what I'm trying to do and tell me if you think it's
possible. For my own edification, I'm trying to implement a C# grammar. I'd
like to implement the pre-processor at the moment. Implementations I've seen
generally using only a lexer and use some type of trick to maintain a stack
(e.g. for nested ifdefs and simple if/elif expressions). I figure why not
use a parser to maintain the stack -- isn't that the reason for existence
for parsers anyway? So that's what I'm trying to do -- use a lexer and
parser to implement the pre-processor.
The big difficulty is changing the lexer rules depending on whether I'm in a
#if def block that is active or not. I figured with ANTLR I'd be able to
compute if the #ifdef block is active and then throw a switch to either
parse tokens and hand those tokens off to the C# parser or consume and
ignore all input up to the next pre-processor instruction thereby disabling
that chunk of code. If I can do this then I could put the pre-processor and
parser in the same file and construct the AST in one pass! Would that be
cool? And clean? And maybe worth making a goal for ANTLR to be able to do?
:)
To be a bit more concrete: Here is the production for matching newline at
the end of pre-processor instructions. The idea would be to enable
PP_SKIPPED_CHARACTERS only if inside a disabling #ifdef block which would
consume all characters till the next pre-processing instruction.
pp_new_line
: SINGLE_LINE_COMMENT? ((NEW_LINE! PP_SKIPPED_CHARACTERS) | EOF!)
;
Here is what I was hoping would work as PP_SKIPPED_CHARACTERS. Unfortunately
I don't seem to understand how to flip lexer rules on and off well enough to
make this work...
PP_SKIPPED_CHARACTERS
: { IfDefedOut }? ( ~(F_NEW_LINE_CHARACTER | F_POUND_SIGN)
F_INPUT_CHARACTER* F_NEW_LINE )*
;
I hope that is enough to give you an idea of what I'm trying to do. This
approach just seems so elegant to me (by which I mean almost all declarative
-- no need to sprinkle procedural logic in among my productions to maintain
a stack or whatever) that I'd hope that it would be do able in ANTLR. What
do you think? Is it a worthy goal? Does it feel possible to you? If not, is
a goal worth trying to achieve?
Thanks,
Chris
On Thu, Jul 28, 2011 at 2:37 PM, Sam Harwell sharw...@pixelminegames.com
wrote:
Hi Chris,
Lookahead prediction occurs before predicates are evaluated. If fixed
lookahead uniquely determines the alternative with a semantic predicate,
the predicate will not be evaluated as part of the decision process. I'm
guessing (but not 100% sure) if you use a gated semantic predicate, then it
will not be entering the rule:
PP_SKIPPED_CHARACTERS
: {false}? = ( ~(F_NEW_LINE_CHARACTER | '#') F_INPUT_CHARACTER*
F_NEW_LINE )*
;
Also, a word of warning: this lexer rule can match a zero-length character
span, which could result in an infinite loop. You should always ensure that
every path through any lexer rule that's not marked fragment will consume
at least 1 character. There's also a bug with certain exceptions in the
lexer that can cause infinite loops - this has been resolved for release 3.4
but I haven't released it yet.
Sam
From: chris king [mailto:kingce...@gmail.com]
Sent: Thursday, July 28, 2011 4:19 PM
To: antlr-interest@antlr.org; Sam Harwell
Subject: Have I found an Antlr CSharp3 lexer bug if...
Have I found an Antlr lexer CSharp3 bug if I can alter program execution
(exception instead of no exception) by introducing a lexer production with a
predicate that is always false? For example
PP_SKIPPED_CHARACTERS
: { false }? ( ~(F_NEW_LINE_CHARACTER | '#') F_INPUT_CHARACTER* F_NEW_LINE
)*
;
I would think that such a production should always be ignored because it's
predicate is always false and therefore would never alter program execution.
Yet I'm seeing a change in the execution of my program. I'm seeing it enter
this function and throw a FailedPredicateException. I wouldn't have expected
that this function should ever even have been executed because the predicate
is always false
[il-antlr-interest: 33368] Re: [antlr-interest] Quoted String Literal - confused by greed=false behavior.
You're reading the greedy option in reverse. :) I'd write the rule this way:
STRING_LITERAL
: ''
( options{greedy=true;}
: ''
| ~''
)*
''
;
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of G. Richard Bellamy
Sent: Wednesday, July 27, 2011 1:49 PM
To: antlr-interest@antlr.org
Subject: [antlr-interest] Quoted String Literal - confused by greed=false
behavior.
I've got a lexer rule that should be gobbling everything after the double
quote '' except for the last double quote - I basically stole the rule from
a post from Jim Idle
(http://www.antlr.org/pipermail/antlr-interest/2010-March/038051.html).
I've also tried other variations on the same rule, and I'm a bit confused as
it seems the {greedy=false;} option is being ignored.
Any help is appreciated
---
INPUT: @(FOO=)
---
lexer grammar Lexer
options
{
language=CSharp3;
TokenLabelType=CommonToken;
}
DQUOTE : '';
STRING_LITERAL
: DQUOTE (options { greedy = false; }
: (
(
{input.LA(1) == '' input.LA(2) == ''}? DQUOTE DQUOTE
| ~DQUOTE
)*
)
)
DQUOTE
;
---
LEXER TRACE (excerpt):
enter STRING_LITERAL line=1:7
enter DQUOTE line=1:7
exit DQUOTE ) line=1:8
enter DQUOTE ? line=1:9
exit DQUOTE ? line=1:9
exit STRING_LITERAL ? line=1:9
line 1:10 mismatched character 'EOF' expecting ''
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33374] Re: [antlr-interest] Quoted String Literal - confused by greed=false behavior.
You have the following rule in your lexer. Remove it:
DQUOTE : '';
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of G. Richard Bellamy
Sent: Wednesday, July 27, 2011 3:20 PM
Cc: antlr-interest@antlr.org
Subject: Re: [antlr-interest] Quoted String Literal - confused by
greed=false behavior.
So, being the confused monkey that I am - Could you help me understand?
If it doesn't have to do with the greedy option, what DOES it have to do
with? I get the same behavior whether using the DQUOTE token or the ''
string literal.
I should mention that I'm using the 3.3.3.8388 CSharp distribution found
here:
http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases
-rb
On 7/27/2011 1:11 PM, Jim Idle wrote:
The rule you show below is not what I posted. Remove the DQUOTE token
and use the literal ''. This has nothing to do with the greedy option.
Jim
-Original Message-
From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
boun...@antlr.org] On Behalf Of G. Richard Bellamy
Sent: Wednesday, July 27, 2011 11:49 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] Quoted String Literal - confused by
greed=false behavior.
I've got a lexer rule that should be gobbling everything after the
double quote '' except for the last double quote - I basically stole
the rule from a post from Jim Idle
(http://www.antlr.org/pipermail/antlr-interest/2010-March/038051.html).
I've also tried other variations on the same rule, and I'm a bit
confused as it seems the {greedy=false;} option is being ignored.
Any help is appreciated
---
INPUT: @(FOO=)
---
lexer grammar Lexer
options
{
language=CSharp3;
TokenLabelType=CommonToken;
}
DQUOTE : '';
STRING_LITERAL
: DQUOTE (options { greedy = false; }
: (
(
{input.LA(1) == '' input.LA(2) == ''}? DQUOTE
DQUOTE
| ~DQUOTE
)*
)
)
DQUOTE
;
---
LEXER TRACE (excerpt):
enter STRING_LITERAL line=1:7
enter DQUOTE line=1:7
exit DQUOTE ) line=1:8
enter DQUOTE ? line=1:9
exit DQUOTE ? line=1:9
exit STRING_LITERAL ? line=1:9
line 1:10 mismatched character 'EOF' expecting ''
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-
email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33379] Re: [antlr-interest] Empty Quoted String Literal
Hi Richard,
The double quote escape character in a verbatim string is , so what you're
actually passing to the parser is a single (unmatched) quote. To pass an
empty string, you'll need to use @.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of G. Richard Bellamy
Sent: Wednesday, July 27, 2011 4:14 PM
To: antlr-interest@antlr.org
Subject: Re: [antlr-interest] Empty Quoted String Literal
Bart,
When I escape the quotes things work on my end as well - you'll note that
I'm passing a set of non-escaped quotes... (C# Verbatim String), so you'll
not be able to test this on your end (since Java doesn't have an equivalent
to the Verbatim String).
Thanks again.
-rb
On 7/27/2011 2:10 PM, Bart Kiers wrote:
Hi Richard,
Both ANTLRWorks' debugger, and my Java test rig:
import org.antlr.runtime.*;
public class Main {
public static void main(String[] args) throws Exception {
WhitespaceInSymbolsLexer lexer = new
WhitespaceInSymbolsLexer(new ANTLRStringStream(\\));
WhitespaceInSymbolsParser parser = new
WhitespaceInSymbolsParser(new CommonTokenStream(lexer));
parser.compileUnit();
}
}
produce no errors when parsing (2 double quotes) as input:
bart@hades:~/Programming/ANTLR/Demos/WhitespaceInSymbols$ java -cp
antlr-3.3.jar org.antlr.Tool WhitespaceInSymbols.g
bart@hades:~/Programming/ANTLR/Demos/WhitespaceInSymbols$ javac
-cp antlr-3.3.jar *.java
bart@hades:~/Programming/ANTLR/Demos/WhitespaceInSymbols$ java -cp
.:antlr-3.3.jar Main
bart@hades:~/Programming/ANTLR/Demos/WhitespaceInSymbols$
Then there must be something going differently in the CSharp3 target
than in the Java target (note that I am not able to test the CSharp3
target here...).
Regards,
Bart.
On Wed, Jul 27, 2011 at 11:01 PM, G. Richard Bellamy
rbell...@pteradigm.com mailto:rbell...@pteradigm.com wrote:
Sam, Bart Jim, I really appreciate your help on this.
Here's a more complete example, without the greedy confusion. I'm
including the combined grammar and a test rig.
I get: CombinedLexer:line 1:2 mismatched character 'EOF'
expecting ''
just before a NullReferenceException.
GRAMMAR:
grammar Combined;
options {
language=CSharp3;
TokenLabelType=CommonToken;
output=AST;
ASTLabelType=CommonTree;
}
@lexer::namespace{StringLiteralLexerTest}
@parser::namespace{StringLiteralLexerTest}
/*
* Parser Rules
*/
public
compileUnit
: STRING
;
/*
* Lexer Rules
*/
STRING : '' ('' | ~'')* '';
TEST RIG:
--
static void Main()
{
CombinedLexer lexer = new CombinedLexer(new ANTLRStringStream(@));
//lexer.TraceDestination = new
ConsoleTextWriter(typeof(CombinedLexer));
CommonTokenStream tokenStream = new CommonTokenStream(lexer);
CombinedParser parser = new CombinedParser(tokenStream);
//parser.TraceDestination = new
ConsoleTextWriter(typeof(CombinedParser));
CommonTree parseTree = parser.compileUnit().Tree;
Console.WriteLine(parseTree.ToStringTree());
}
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33334] Re: [antlr-interest] To Sam Harwell
Hi Chris, I tried this once in the past, but ran into a problem. Months later I found out the problem was in the order in which I included things in my project file, and I updated the directions to correct it. Obviously I forgot to update my own projects based on the new information. :) Thanks for bringing this to my attention. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of chris king Sent: Monday, July 25, 2011 6:39 PM To: antlr-interest@antlr.org Subject: [antlr-interest] To Sam Harwell Sam, hey, hope this finds you. Very small suggestion follows :). In Antlr3.StringTemplate.csproj could you use AntlrBuildTaskPath$(MSBuildProjectDirectory)\..\bin\Bootstrap/AntlrBuildT askPath to reference the bootstrap directory? Originally it used the solutionDir and that prevented me from including a subset of the projects in my project (so I could simply reference them and have all the debugging, pdb, source, etc just work). Thanks, Chris List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33279] Re: [antlr-interest] Left recursive grammar
Your example is ambiguous as well as left recursive. I assume you meant one
of the following:
a : a B | A;
a : A* B | A;
The first can be written as:
a : A (B^)*;
The second can be written as
a : A (A* B^)? | A | B;
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Luigi Iannone
Sent: Thursday, July 21, 2011 8:00 AM
To: ANTLR
Subject: [antlr-interest] Left recursive grammar
Hi all,
I have this simple grammar
grammar test;
options {
language = Java;
output = AST;
}
a
:
a*B -^(B a*)
| A
;
B :
'.B'
;
A :
'A'
;
and I get the following output when I try to generate the parser in
ANTRLWorks
[13:48:53] error(210): The following sets of rules are mutually
left-recursive [a]
I read on the Web that there are solutions to solve this, however they will
mess up the associativity, which I need to keep instead.
So, for instance, for the input
A.B.B
the AST tree should be
^(B ^(B A))
Is there any way to change the grammar in order to eliminate the left
recursion and obtain the above tree. I am afraid I do not get how to do it
by just looking at what is there online about left recursive grammars.
Thanks a lot for your help,
Luigi
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33284] Re: [antlr-interest] ANTLR gives segmentation fault for very large input
To skip the AST, just don't use the output=AST option. Here are some specs on the tokens. I'm including the overhead of having them in a CommonTokenStream (or equivalent) because they're not very useful otherwise. Java target, 32-bit VM: 48 bytes/token. Java target, 64-bit VM: 64 bytes/token. CSharp3 target: Same as Java target. C target, 32-bit: 148 bytes/token. C target, 64-bit: 248 bytes/token. You have 6 tokens per line, and it sounds like you're using the C target. The small/large files use 39KiB/3.72GiB of memory respectively for the tokens on a 32-bit machine. They use 65.4KiB/6.24GiB on a 64-bit machine. I'm developing an alternative to CommonToken that uses 8 bytes/token in all of the above targets. Once it's ready (which may not be until ANTLR v4), the same files will only need 2.1KiB/206MiB of memory, a savings of 94.6% on the 32-bit C target, and nearly 97% on the 64-bit C target. -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Piyush Sent: Thursday, July 21, 2011 10:18 AM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] ANTLR gives segmentation fault for very large input Is there is any way to delete AST (Abstract Syntax Tree) because it is of no use for my work. On Tue, Jul 19, 2011 at 9:08 PM, Jim Idle [via ANTLR] ml-node+6599207-454424018-346...@n2.nabble.com wrote: You are running out of memory - split up the input in some sensible way. Jim -Original Message- From: [hidden email] [mailto:antlr-interest- [hidden email]] On Behalf Of Piyush Sent: Tuesday, July 19, 2011 1:51 AM To: [hidden email] Subject: [antlr-interest] ANTLR gives segmentation fault for very large input Sir when i am trying to parse a very big input file (of nearly 450 lines) ANTLR is giving segmentation fault. Just for example my grammar funny.g is parsing input file input.v(of near about 45 lines) but gives segmentation fault for big_file_input.v(of about 450 lines) ,which also contains the same input as of input.v 10 times I am attaching my grammar (funny.g and input files) below. So please help me out where i am doing wrong or this is antlr's bug? Thanking You Piyush http://antlr.1301665.n2.nabble.com/file/n6598011/fun.tar fun.tar -- View this message in context: http://antlr.1301665.n2.nabble.com/ANTLR- gives-segmentation-fault-for-very-large-input-tp6598011p6598011.html Sent from the ANTLR mailing list archive at Nabble.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address If you reply to this email, your message will be added to the discussion below: http://antlr.1301665.n2.nabble.com/ANTLR-gives-segmentation-fault-for- very-large-input-tp6598011p6599207.html To unsubscribe from ANTLR gives segmentation fault for very large input, click here. Cheers! Piyush Bengal Engineering Science University Computer Science and Technology -- View this message in context: http://antlr.1301665.n2.nabble.com/ANTLR-gives-segmentation-fault-for-very-large-input-tp6598011p6607198.html Sent from the ANTLR mailing list archive at Nabble.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33218] Re: [antlr-interest] CSharp3 target with 3.3.3 and extended AST Operators (or not).
Were you generating code with the C# port of the tool (Antlr3.exe) or with the Java version? I've only tested this feature with the C# version. I believe if you use the Java version to generate it, you have to explicitly state the terminal option. The C# port defaults to this option, but the Java version defaults to a different option. between_number: number DASHttype=BETWEEN^ number ; Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of G. Richard Bellamy Sent: Monday, July 18, 2011 12:40 PM To: antlr-interest@antlr.org Subject: [antlr-interest] CSharp3 target with 3.3.3 and extended AST Operators (or not). I've been trying to use the new operators for AST Node type conversion without rewrite rules. Attempt (doesn't work for me): between_number: : number DASHBETWEEN^ number ; Works: between_number: : n1=number DASH n2=number - ^(BETWEEN $n1 $n2) ; Am I perhaps missing something related to implicit lists (since `number` appears twice)? List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33202] Re: [antlr-interest] Hi! about making a target for ANTLR
That's what it does now...? Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Douglas Godfrey Sent: Saturday, July 16, 2011 10:13 AM To: Terence Parr Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Hi! about making a target for ANTLR How about a generalized Antlr Target builder based on a TreeParser grammar that processes the AST generated by Antlr that uses String Templates to emit the source code for the target language. The task of building a new target then is reduced to completing the String Template productions and implementing the Antlr classes for the target language. On Fri, Jul 15, 2011 at 1:55 PM, Terence Parr pa...@cs.usfca.edu wrote: for what language? Ter On Jul 15, 2011, at 3:56 AM, ante...@freemail.hu wrote: Hi, If I made a target for ANTLR, and I am done with it. WHat should I do next? Regards Márton Papp List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33151] Re: [antlr-interest] Tree pattern won't rewrite AST C#
In your partial class file that accompanies your grammar, you should add the
following function override:
partial class MyTreeParser {
public override IAstRuleReturnScope Bottomup() { return bottomup(); }
}
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of DJB MASTER
Sent: Tuesday, July 12, 2011 10:38 PM
To: antlr-interest@antlr.org
Subject: [antlr-interest] Tree pattern won't rewrite AST C#
Hey, I'm trying to apply simple tree patterns that can remove redundancy in
multiplication expressions, such as x*0 - 0. However I can't get it to
rewrite my AST I feed in.
I'm using again
http://media.pragprog.com/titles/tpdsl/code/IR/Vec/VecMathAST.g
and I'm using this as my simplifier
http://media.pragprog.com/titles/tpdsl/code/walking/patterns/Simplify.g.
I'm getting a correct AST because it's identical to what Terrance gets in
the LP book.
I'm using this code to call my simplifier...
CommonTree AST = (CommonTree)parser.statlist().Tree;
CommonTreeNodeStream nodes = new CommonTreeNodeStream(AST);
nodes.TokenStream = tokens;
Simplifier simplifier = new Simplifier(nodes); CommonTree AST =
(CommonTree)simplifier.Downup(AST, true);
But it seems to do nothing.
--
View this message in context:
http://antlr.1301665.n2.nabble.com/Tree-pattern-won-t-rewrite-AST-C-tp657761
4p6577614.html
Sent from the ANTLR mailing list archive at Nabble.com.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33142] Re: [antlr-interest] desperate noob
Also, make sure you back up a copy of the generated file you have. You don't want to accidentally generate some code over the top of it only to find out the new grammar wasn't yet complete. :) -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of The Researcher Sent: Tuesday, July 12, 2011 3:55 PM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] desperate noob Hi Louis, In the comments of the source code for each rule should be the original rule from the grammar. I don't know of any tool in or out of ANTLR that will automate the task so you will have to extract it by hand. HTH Eric List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33100] Re: [antlr-interest] ASTLabelType C# Problem
You should throw a NotImplementedException instead of returning null. The rewrite exception indicates a mismatch between the matching and rewrite portions of the rules in your grammar. Typically that particular message appears when you have a rule written like one of the following: Example 1: foo : bar - bar; bar : x*; In the above case, the rule bar won't create a tree if 0 instances of x are matched. The rule foo should actually be written foo : bar-bar?; to handle this. Example 2: foo : bar? - bar; In the above case bar might again be missing, so you should write foo : bar? - bar?;. Example 3: foo : bar - bar; bar : x; x : y!; In the above case, the rule x doesn't return a tree because it explicitly ignores the result of y. Since rule bar doesn't have a rewrite rule associated with it, it implies bar : x - x?; which works fine. The net result in foo is similar to example 1 above. Another way to handle this is simply remove the explicit rewrite from foo to make foo : bar; Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of DJB MASTER Sent: Sunday, July 10, 2011 8:18 PM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] ASTLabelType C# Problem Ok, Got all that, but now I'm getting a RewriteEmptyStreamException on this line... 'adaptor.AddChild(root_1, stream_expression.NextTree());'. A soon as I fix something, something else goes wrong lol. BTW, i'm just returning null from ErrorNode(..) for the moment, although I'm passing in valid code to my parser so it shouldn't need any error nodes. -- View this message in context: http://antlr.1301665.n2.nabble.com/ASTLabelType-C-Problem-tp6567453p6569254. html Sent from the ANTLR mailing list archive at Nabble.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33094] Re: [antlr-interest] ASTLabelType C# Problem
I'm not sure which templates are in use by AntlrWorks 1.4.2, but they certainly aren't the latest/correct version for the CSharp3 target. Under most conditions, the CSharp3 target doesn't emit the *_return classes at all, and renaming the rules wouldn't affect that. I suggest trying the latest version of the C# target available on the following page: http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of DJB MASTER Sent: Sunday, July 10, 2011 5:17 PM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] ASTLabelType C# Problem I've generated the code with AntlrWorks 1.4.2. and also used the antlr tool, both give the same compile errors. The reason that 'statement_return' doesn't exist in your version, is i've just renamed the rules in the grammar from 'stat' to 'statement', no big deal that should effect the generated code. The parser generates a CommonTree AST just fine. This isn't specific to this grammar. I wrote another grammar, which again generates an AST fine using CommonTree, but when I add ASTLabelType to change the node type, I get these errors. -- View this message in context: http://antlr.1301665.n2.nabble.com/ASTLabelType-C-Problem-tp6567453p6568771. html Sent from the ANTLR mailing list archive at Nabble.com. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33096] Re: [antlr-interest] ASTLabelType C# Problem
If your parser uses a special tree type, you should implement the following
method in your helper file to ensure that there's never a case where the
user forgets to set the tree adaptor.
partial void CreateTreeAdaptor(ref ITreeAdaptor adaptor) {
adaptor = new MyTreeAdaptor(this);
}
You'll need to implement a tree adaptor for the new node type, which at a
minimum should implement the following methods:
internal class MyTreeAdaptor : CommonTreeAdaptor {
public override object Create(IToken payload) {
return new MyTreeNodeType(payload);
}
public override object ErrorNode(ITokenStream input, IToken start,
IToken stop, RecognitionException e) {
return new MyTreeNodeErrorType(input, start, stop, e);
}
}
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of DJB MASTER
Sent: Sunday, July 10, 2011 6:05 PM
To: antlr-interest@antlr.org
Subject: Re: [antlr-interest] ASTLabelType C# Problem
Ok, I downloaded the preview release and it has seemed to fix those errors.
I was using the stable release because I didn't want to use a library that
was 'temperamental'.
Just one more thing, now when using my ASTNode I get an invalid cast from
CommonTree to ASTNode in my rules. On these lines...
'Number16_tree = (ASTNode)adaptor.Create(Number16);'
I've set the adaptor using...
'ASTAdaptor treeAdaptor = new ASTAdaptor(); parser.TreeAdaptor =
treeAdaptor;'
Do I have to set the adaptor somewhere else? I've seen people talking about
invalid casts from CommonTree, not sure what's required.
Thanks again.
--
View this message in context:
http://antlr.1301665.n2.nabble.com/ASTLabelType-C-Problem-tp6567453p6568868.
html
Sent from the ANTLR mailing list archive at Nabble.com.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 33086] [antlr-interest] Updated Visual Studio 2010 Extensions for ANTLR and ST4
Hello everyone, I just published updated versions of my Visual Studio 2010 extensions to the Visual Studio Gallery. Below is a note for existing users followed by a description of the extensions and links to the Visual Studio Gallery where they are published. Important: If you previously installed an early copy of the following 3 extensions, you'll need to manually uninstall them (all of them) from the Tools - Extension Manager. dialog before installing the new version. In the future, the extensions should update correctly without having to reinstall them. . ANTLR language support . StringTemplate 4 language support . Visual Studio Extensibility Framework With that out of the way, here is a description of the various extensions. Screenshots are available on the gallery pages for each extension. ANTLR Language Support http://visualstudiogallery.msdn.microsoft.com/25b991db-befd-441b-b23b-bb5f8d 07ee9f . Provides syntax highlighting and basic IntelliSense features for ANTLR v3 grammars. . Adds the exceptions from Antlr.Runtime and Antlr.Runtime.Tree to the Debug-Exceptions... dialog for easy configuration. StringTemplate 4 Language Support http://visualstudiogallery.msdn.microsoft.com/5ca30e58-96b4-4edf-b95e-3030da f474ff . Provides syntax highlighting and a select few IntelliSense features for StringTemplate files. . Some features are limited to the default delimiters '' and ''. The following extensions are not quite as developed as the two above, but I'll include them since some people may find them useful and because I used ANTLR to create them. All of these extensions are being created for one of my ongoing research projects. Java Language Support http://visualstudiogallery.msdn.microsoft.com/bc561769-36ff-4a40-9504-e266e8 706f93 PHP Language Support http://visualstudiogallery.msdn.microsoft.com/2a10ba81-26c5-47d9-939b-6bcc7b bec251 Alloy Language Support (http://alloy.mit.edu/community) http://visualstudiogallery.msdn.microsoft.com/4ae31812-f841-45c3-8052-113077 5473f8 Go Language Support (http://golang.org) http://visualstudiogallery.msdn.microsoft.com/bd7675ba-1bf5-4395-8c5a-4fc19d fc0d76 Thank you, Sam Harwell List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32946] Re: [antlr-interest] ANTLR problem with .+ in tree grammar
I use the following rule for that:
genericBlock
: '{'
(genericBlock | ~('{'|'}'))*
'}'
;
Since the curly braces are inside the genericBlock rule, you'll have to modify
your componentStatement rule:
componentStatement
:'component'^ IDENT genericBlock ';'!
;
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Dariusz Dwornikowski
Sent: Saturday, June 25, 2011 5:30 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] ANTLR problem with .+ in tree grammar
Hi,
I've got such a problem. I am experimenting with the ANTLR in order to
implement some kind of language later. The example below.
Grammar file (interesting part):
...
tokens{
BODY;
}
componentStatement
:'component'^ IDENT '{'! body '}'! ';'!
;
body
:.+ - BODY
;
...
tree gramar file:
...
component
: ^('component' name=IDENT body=BODY) {comp.add(new
Component($name.text, $body.text));}
;
..
What i want is to catch such a situation (for experiment purposes only):
component Foo {
anything goes here 123 :)
};
Now, I know rewrite in my example rewrites .+ to BODY it is ok. When I change
the grammar to body
:.+ - BODY
;
and tree grammar to:
component
: ^('component' name=IDENT body=(.+)) {comp.add(new
Component($name.text, $body.text));}
;
the generates Walker generates it wrong:
comp.add(new Component((name!=null?name.getText():null), b.text)); it should
have been b.getText()
How can I change files in order to catch what I want?
PS: ANTLR is great:) And Greetings to everybody :)
--
Pozdrawiam,
-
Dariusz Dwornikowski,
Institute of Computing Science, Poznań University of Technology
www.cs.put.poznan.pl/ddwornikowski/
room 421 WE | tel. +48 61 665 21 24 | mobile: +48 533 890 600
-
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32737] Re: [antlr-interest] [antlr-dev] Need help with tree traversing in ANTLR 3.3!
Hi Payam,
Any rule in a parser or tree parser can return 0 or more items. Since the
Java method implementing a rule can only return 0 or 1 items, some special
handling is in place to allow you to do what you're after. Consider a rule
foo which currently returns a single int:
foo returns [int x] : {$x = 3;} ;
In a regular parser (non-AST), the rule will be generated as:
private int foo();
When you add the output=AST option, the rule now returns 4 values:
options {
output=AST;
ASTLabelType=CommonTree;
LabelType=CommonToken;
}
1. int x
2. ASTLabelType tree
3. LabelType start
4. LabelType stop
The function is generated as follows:
foo_return foo();
Which you can call like this to get the return values:
foo_return result = parser.foo();
int x = result.x;
CommonTree tree = result.tree;
Sam
On Jun 10, 2011, at 10:01 AM, Payam Fard wrote:
Dear all,
I have a grammar for a fairly small and simple language that I have
created for our project. Once every command in this language is parsed
through this grammar, a Java object is returned. Everything is working fine.
Now, I have a requirement to read all the available rules/commands of the
language (I am only interested in the command/rule names). In order to do
so, I have been trying to get the AST tree working by adding options to my
grammar for AST tree. For some reason, when I add these options to my
grammar, I am getting compilation error complaining that my grammar rules
cannot return any Java objects! Can I have commands/rules of my grammar
return some Java objects and at the same time be able to build an AST tree
off of my grammar?
Is there an easy way to read these grammar command/rule names using antlr
runtime API's?
Any code sample out there that shows how do to the above (namely returning
objects from grammar rules and building the tree at the same time)?
Any help here would be greatly appreciate.
___
antlr-dev mailing list
antlr-...@antlr.org
http://www.antlr.org/mailman/listinfo/antlr-dev
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32709] Re: [antlr-interest] [CSharp3] rule visibility in composite grammars
I'm also very interested in ways to make ANTLR grammars
target-language-agnostic. I know ANTLR version 4 will provide more
consistent semantics across a number of language features, but I'm not sure
if a specific symbol table support like you mention will be included. The
problem with such a feature is it increases the complexity of the grammar
language specification, increases the size of the runtime and/or the
complexity of the code generation templates (making things more difficult
for target developers), and only meets the semantic language features of a
small number of users.
One idea I had is for a target-agnostic call syntax that could be used in
semantic predicates and actions. It could use a form such as the following:
@{FunctionName(arg1,arg2,arg3)}
Where an argument can be one of the following:
. A reference to a token, rule return value, or label in the rule.
. A reference to an argument passed to the rule.
. A reference to a value in an attribute scope.
The target would then declare a user-definable method with the appropriate
parameter types. The Java and CSharp2 targets could create an abstract
method, the CSharp3 target could create a partial method, and a C/C++
targets could declare the method in a generated header.
Sam
From: Douglas Godfrey [mailto:douglasgodf...@gmail.com]
Sent: Tuesday, June 07, 2011 8:09 AM
To: Ranco Marcus
Cc: Sam Harwell; antlr-interest@antlr.org
Subject: Re: [antlr-interest] [CSharp3] rule visibility in composite
grammars
2 months ago I submitted a feature request for an Antlr built-in symbol
table to support the common
requirements of the majority of block-structured languages. By making the
SymbolTable part of the
Antlr grammar language the interface can be much cleaner. The implementation
of the SymbolTable
classes would be part of the target runtime(s).
i.e.
new_variable_name:
(Identifier.IsNewSymbol()) = Identifier.AddSymbol();
SymbolTable - NameSpace - SymbolScope - Symbol - Attribute-List {
optional for structs - NameSpace }
On Tue, Jun 7, 2011 at 5:56 AM, Ranco Marcus ranco.mar...@epirion.nl
wrote:
Yes, that sounds like a good idea. I would definitely be in favour of
merging the grammars before generating the code. My only concern is that
this approach would deviate from the general ANTLR approach.
In general, I have found ANTLR to be a great tool for parser generation, but
never really liked the way target specific actions are mixed with the
grammar definition. Ideally, I would like my grammars to be _completely_
target agnostic (no actions, no visibility modifiers, members, headers,
superClass definitions, etc.) and have an abstract mechanism that we can use
to hookup actions and implementation specific stuff to the generated
grammar.
Do you know if there are plans to redesign the composite grammar feature in
v4?
Best regards,
Ranco
-Original Message-
From: Sam Harwell [mailto:sharw...@pixelminegames.com]
Sent: Sunday, May 29, 2011 11:08 AM
To: Ranco Marcus; antlr-interest@antlr.org
Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
grammars
I'm not going to be able to address this issue until the second week of
June.
That said, it seems the best way to handle all these issues with delegate
grammars is to inline their rules before code generation. Suppose you have
grammar C importing A and B, and you also have D importing A and B. The
code generation will result in classes C, C_A, C_B, D, D_A, and D_B.
Clearly
the independent generation of C_A and D_A during code generation does
not allow a single instance of the imported A grammar to be shared by C
and
D. If we instead flatten the imported grammar hierarchy and only
generate
classes C and D, then everything behaves like it was written in a single
grammar. Do you see any immediate problems with this potential approach?
-Original Message-
From: Ranco Marcus [mailto:ranco.mar...@epirion.nl]
Sent: Wednesday, May 25, 2011 4:26 PM
To: Sam Harwell; antlr-interest@antlr.org
Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
grammars
Hi Sam/all,
When a (tree) grammar C imports (tree) grammars A and B, where grammar
A calls a rule R from grammar B, a call is being made from delegate parser
C_A
to a delegate rule R (targeting C_B) in the composite parser C (its
parent).
Now that the visibility of the delegate rules in C match the visibility of
the
imported grammar, the rule R has to be made public for the above to work.
In our grammars, we build up internal structures that are subsequently
processed. In our case, that means that all those internal structures have
to
be made public as well. This could be solved by allowing ANTLR rules to
have
'internal' visibility. Also, imported grammars can probably remain
internal as
well.
What are your thoughts on this?
Thanks, Ranco
-Original Message-
From: Sam Harwell [mailto:sharw
[il-antlr-interest: 32603] Re: [antlr-interest] [CSharp3] rule visibility in composite grammars
I'm not going to be able to address this issue until the second week of June. That said, it seems the best way to handle all these issues with delegate grammars is to inline their rules before code generation. Suppose you have grammar C importing A and B, and you also have D importing A and B. The code generation will result in classes C, C_A, C_B, D, D_A, and D_B. Clearly the independent generation of C_A and D_A during code generation does not allow a single instance of the imported A grammar to be shared by C and D. If we instead flatten the imported grammar hierarchy and only generate classes C and D, then everything behaves like it was written in a single grammar. Do you see any immediate problems with this potential approach? -Original Message- From: Ranco Marcus [mailto:ranco.mar...@epirion.nl] Sent: Wednesday, May 25, 2011 4:26 PM To: Sam Harwell; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars Hi Sam/all, When a (tree) grammar C imports (tree) grammars A and B, where grammar A calls a rule R from grammar B, a call is being made from delegate parser C_A to a delegate rule R (targeting C_B) in the composite parser C (its parent). Now that the visibility of the delegate rules in C match the visibility of the imported grammar, the rule R has to be made public for the above to work. In our grammars, we build up internal structures that are subsequently processed. In our case, that means that all those internal structures have to be made public as well. This could be solved by allowing ANTLR rules to have 'internal' visibility. Also, imported grammars can probably remain internal as well. What are your thoughts on this? Thanks, Ranco -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: Thursday, April 07, 2011 3:57 PM To: Ranco Marcus; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars The visibility of delegated rules now matches the way they're declared in the imported grammar. Since some of them return nested classes declared in the delegate grammar, I had to make the delegate grammar 'public' and all its rules 'internal'. I fixed the reserved names issue right after releasing 3.3.1, so it'll be included when I release 3.3.2. Note that rule parameters and return values need to include an explicit '@' in their declarations in the grammar, as follows. Labels and rule names do not need the '@'. namespace[int @in] returns [int @out] : as=A B; Sam -Original Message- From: Ranco Marcus [mailto:ranco.mar...@epirion.nl] Sent: Thursday, April 07, 2011 6:49 AM To: Sam Harwell; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars Sounds great, Sam! What will be the visibility of the methods in the 'Delegated rules' region now that the class of the imported grammar is internal? Private or internal? Another small suggestion is to prefix variable names (at least those that are reserved in C#) with an at-sign. The generated class for the grammar below has a compile error since 'as' is not a valid variable name while '@as' is. name : as=firstName lastName; Are you planning to release new binaries any time soon? Thanks again, Ranco Marcus -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: woensdag 6 april 2011 17:20 To: Ranco Marcus; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars Hi Ranco, I made the following changes so far: * An imported grammar is always generated as an internal class instead of a public class. * Rules within the imported grammar are always declared public since they are only called by the root grammar. How does that sound? Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ranco Marcus Sent: Wednesday, March 23, 2011 6:40 AM To: antlr-interest@antlr.org Subject: [antlr-interest] [CSharp3] rule visibility in composite grammars Hi all/Sam, While using the CSharp3 target on a composite grammar, I came across a few issues regarding rule visibility. If no modifier (private, protected, public, ...) is specified for a rule, the Java target generates public methods (since it is not implemented IIRC) and the CSharp3 target generates private methods. I like the idea to explicitly specify which rules are callable (by making them public). However, for composite grammars the generated parser contains delegate rules for every rule of the imported grammar, even for rules in the imported grammar that aren't public (causing compile errors). This can solved this adding a public modifier to every rule of the imported grammar, but I was wondering if there's a more
[il-antlr-interest: 32490] Re: [antlr-interest] ANTLR C# port
My initial guess is the exceptions are being thrown in the lexer. By default, the implementation of NextToken in the lexer catches any RecognitionException and handles it by skipping the invalid token. Custom handling of the exception depends on exactly what you're trying to accomplish. One way to start is adding the following rule to the end of your lexer, which passes the invalid tokens to the lexer for handling. INVALID_CHAR : . ; Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Dilip Ranganathan Sent: Wednesday, May 18, 2011 3:00 AM To: antlr-interest@antlr.org Subject: [antlr-interest] ANTLR C# port Hi I have been banging my head with Sam Harwell's Antlr C# port since morning. SO far I have managed to integrate grammar compilation with Visual Studio 2010 following the instructions on this page: http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases However I am stuck with something for the moment. A simple test driver program I wrote to test the generated lexer/parser doesn't report any kind of error when I pass invalid input. That is, the generated parser code is eating the exception and the output window only shows that a few first chance exceptions were generated. Now I added these exceptions to Debug-Exceptions list in VS 2010 (under CLR) and that way I was able to make the debugger break into the offending code but I can't believe this is the way to do it? Shouldn't I be seeing something on the console or at least as an exception I must be able to catch and display exactly what was wrong with my input? List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32497] Re: [antlr-interest] ANTLR C# port
After the debugger stops where the exception is thrown, you can use the Step Over command to step over each catch and finally block until the exception is handled. Can you check this and let me know which catch block is actually handling the exception? From: Dilip Ranganathan [mailto:misc.us...@gmail.com] Sent: Friday, May 20, 2011 3:58 AM To: Sam Harwell Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] ANTLR C# port I may have miscommunicated something here. I have a very simple Hello World! style grammar that I wrote to learn ANTLR. I deliberately fed the lexer/parser wrong input to see what happens. I can see this exception being caught by the parser (I forced the debugger to break into this exception by explicitly listing this exception under Debug-Exceptions list in VS2010). But the outer client code that fires off the parser (using ANTLRStringStream and friends) is bracketed by try/catch blocks but never gets anything. This probably makes sense if library code is eating those exceptions but if it does, how can client code know something went wrong? On Thu, May 19, 2011 at 5:52 AM, Sam Harwell sharw...@pixelminegames.com wrote: My initial guess is the exceptions are being thrown in the lexer. By default, the implementation of NextToken in the lexer catches any RecognitionException and handles it by skipping the invalid token. Custom handling of the exception depends on exactly what you're trying to accomplish. One way to start is adding the following rule to the end of your lexer, which passes the invalid tokens to the lexer for handling. INVALID_CHAR : . ; Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Dilip Ranganathan Sent: Wednesday, May 18, 2011 3:00 AM To: antlr-interest@antlr.org Subject: [antlr-interest] ANTLR C# port Hi I have been banging my head with Sam Harwell's Antlr C# port since morning. SO far I have managed to integrate grammar compilation with Visual Studio 2010 following the instructions on this page: http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases However I am stuck with something for the moment. A simple test driver program I wrote to test the generated lexer/parser doesn't report any kind of error when I pass invalid input. That is, the generated parser code is eating the exception and the output window only shows that a few first chance exceptions were generated. Now I added these exceptions to Debug-Exceptions list in VS 2010 (under CLR) and that way I was able to make the debugger break into the offending code but I can't believe this is the way to do it? Shouldn't I be seeing something on the console or at least as an exception I must be able to catch and display exactly what was wrong with my input? List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32442] Re: [antlr-interest] meaning of #xxx
I think it translates to the following in ANTLR 3:
sql_literal :
(NUMBER | QUOTED_STRING | 'null')
- ^(SQL_LITERAL[sql_literal] NUMBER? QUOTED_STRING? 'null'?)
;
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Philippe Frankson
Sent: Thursday, May 12, 2011 9:51 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] meaning of #xxx
Hi,
In a grammar done with ANTLR 2.x, I have seen the following expression:
sql_literal:
( NUMBER | QUOTED_STRING | null )
{ #sql_literal = #([SQL_LITERAL, sql_literal], #sql_literal); }
;
Anybody can explain the meaning of the code inside the { } ? (or send a
link where I can find the doc).
#sql_literal ?
#(x,y) ?
I could not find relevant documentation explaining that.
Thx a lot.
Philippe
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32277] Re: [antlr-interest] [C#] Lexer Not Throwing Errors
I'm not sure why it differs from the Java target. The nextToken()
implementation in the Java target appears to use the same exception handling
that the CSharp3 target uses.
Sam
From: Justin Holewinski [mailto:justin.holewin...@gmail.com]
Sent: Thursday, April 21, 2011 12:49 PM
To: Sam Harwell
Cc: antlr-interest@antlr.org
Subject: Re: [antlr-interest] [C#] Lexer Not Throwing Errors
On Thu, Apr 21, 2011 at 1:06 PM, Sam Harwell sharw...@pixelminegames.com
wrote:
Hi Justin,
The exception does get thrown, but is handled in Lexer.NextToken() by
tossing out a character and continuing. There are many ways to deal with
this situation ranging from overriding NextToken in your lexer to creating
an ANYCHAR rule at the very end that allows the incorrect input characters
to pass on to the parser for handling there.
ANYCHAR : . ;
I see. Out of curiosity, any reason why this is different from the bahavior
of the Java target?
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Justin Holewinski
Sent: Tuesday, April 19, 2011 11:49 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] [C#] Lexer Not Throwing Errors
I'm using the C# 3 port of Antlr (target and code generator) and I'm
experiencing a lack of lexer errors being generated. As a concrete example,
consider the following grammar:
grammar CSTest;
public
compilation_unit
: IDENTIFIER* EOF
;
IDENTIFIER
: (ID_PART) (ID_PART_OR_NUMBER)*
;
fragment
ID_PART
: ('a'..'z' | 'A'..'Z' | '_')
;
fragment
ID_PART_OR_NUMBER
: ID_PART
| '0'..'9'
;
WS
: (' '|'\r'|'\t'|'\n') { /*Skip();*/ $channel=Hidden; }
;
The only production in this grammar should allow for an arbitrary sequence
of identifiers, and it does. However, it also accepts ALL input, including
binary files and anything else I pipe into it. Is the default in the C#
target to silently ignore any lexing errors? In my driver program, I am not
seeing any throw exceptions when I call the compilation_unit rule.
using System;
using System.IO;
using Antlr.Runtime;
namespace CSTest {
public class Program {
public static void Main(string[] args) {
string filename;
filename = args[0];
if(!Path.IsPathRooted(filename)) {
filename = Path.Combine(Environment.CurrentDirectory, filename);
}
ICharStream input = new ANTLRFileStream(filename);
CSTestLexer lexer = new CSTestLexer(input);
CommonTokenStream tokens = new CommonTokenStream(lexer);
CSTestParser parser = new CSTestParser(tokens); try {
parser.compilation_unit(); } catch(RecognitionException re) {
Console.Out.WriteLine(EXCEPTION);
Console.Out.WriteLine(re.StackTrace);
}
}
}
}
--
Thanks,
Justin Holewinski
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
Thanks,
Justin Holewinski
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32168] Re: [antlr-interest] throwing non-antlr exceptions
Hi Joe,
This feature is not yet implemented in v3. I may be able to get it
implemented for release 3.4, but there are some technical issues that
prevent it from working in a few cases that make it difficult.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Joe Lemmer
Sent: Tuesday, April 12, 2011 6:59 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] throwing non-antlr exceptions
Hi there,
I'd like to throw a Non-Antlr exception from one of my rules in a Parser.
I'm using ANTLR v3
I've tried to find some documentation and came across some stuff for ANTLR
v2 that says the following:
To specify that your parser (or
tree parser rule) can throw a non-ANTLR specific exception, use the
exceptions clause. For example, here is a simple parser specification with
a rule that throws MyException:
class P extends Parser;
a throws MyException
: A
;
ANTLR generates the following for rule a:
public final void a()
throws RecognitionException,
TokenStreamException,
MyException
{
try {
match(A);
}
catch (RecognitionException ex) {
reportError(ex);
consume();
consumeUntil(_tokenSet_0);
}
}
I have implemented my rule as follows:
variable throws IdentifierIsKeywordException @after {
if($i.text.equals(repeat)) { throw new
IdentifierIsKeywordException(\repeat\is a keyword, and may not be used as
an identifier.);} }
:i=IDENTIFIER - IDENTIFIER
;
but in the java file that is created for my parser, the method relating to
this rule still only throws the RecognitionException.
I really want to throw this exception and for it to be propagated right down
to the class that calls my Parser. I'd be grateful for any hints as to where
I'm going wrong and how I can get the method in the parser for my variable
rule to thow my IdentifierIsKeywordException.
Many thanks in advance
Joe
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32127] Re: [antlr-interest] [CSharp3] rule visibility in composite grammars
The visibility of delegated rules now matches the way they're declared in the imported grammar. Since some of them return nested classes declared in the delegate grammar, I had to make the delegate grammar 'public' and all its rules 'internal'. I fixed the reserved names issue right after releasing 3.3.1, so it'll be included when I release 3.3.2. Note that rule parameters and return values need to include an explicit '@' in their declarations in the grammar, as follows. Labels and rule names do not need the '@'. namespace[int @in] returns [int @out] : as=A B; Sam -Original Message- From: Ranco Marcus [mailto:ranco.mar...@epirion.nl] Sent: Thursday, April 07, 2011 6:49 AM To: Sam Harwell; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars Sounds great, Sam! What will be the visibility of the methods in the 'Delegated rules' region now that the class of the imported grammar is internal? Private or internal? Another small suggestion is to prefix variable names (at least those that are reserved in C#) with an at-sign. The generated class for the grammar below has a compile error since 'as' is not a valid variable name while '@as' is. name : as=firstName lastName; Are you planning to release new binaries any time soon? Thanks again, Ranco Marcus -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: woensdag 6 april 2011 17:20 To: Ranco Marcus; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars Hi Ranco, I made the following changes so far: * An imported grammar is always generated as an internal class instead of a public class. * Rules within the imported grammar are always declared public since they are only called by the root grammar. How does that sound? Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ranco Marcus Sent: Wednesday, March 23, 2011 6:40 AM To: antlr-interest@antlr.org Subject: [antlr-interest] [CSharp3] rule visibility in composite grammars Hi all/Sam, While using the CSharp3 target on a composite grammar, I came across a few issues regarding rule visibility. If no modifier (private, protected, public, ...) is specified for a rule, the Java target generates public methods (since it is not implemented IIRC) and the CSharp3 target generates private methods. I like the idea to explicitly specify which rules are callable (by making them public). However, for composite grammars the generated parser contains delegate rules for every rule of the imported grammar, even for rules in the imported grammar that aren't public (causing compile errors). This can solved this adding a public modifier to every rule of the imported grammar, but I was wondering if there's a more convenient way of doing this. Furthermore, the methods for the delegate rules in the composite grammars are all public, regardless of the visibility of the rule in the imported grammar. IMHO, it would be better to make them private and use the composite grammar to explicitly specify the rules that are externally visible. What are your thoughts on this? Btw, the CSharp3 templates are also included in the original Java tool. Does that mean that we don't need the .NET version of the tool anymore? Best regards, Ranco Marcus Epirion Knowledge Solutions B.V. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32107] Re: [antlr-interest] Q: how to incorporate a preprocessor in the flow?
This still works using the method I described. Whenever I encounter an identifier, I process the next token using the mechanism I described earlier, and if necessary substitute a new token when combines the two identifiers. If the next token (or tokens if they are the result of another preprocessor substitution) is not another identifier, then I push them to a queue so I don't have to process them again. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of A Z Sent: Tuesday, April 05, 2011 1:16 AM To: Martin d'Anjou Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Q: how to incorporate a preprocessor in the flow? I tried that approach when I first started with ANTLR but had difficulty handling arbitrary token rearrangement. Early on I couldn't figure out how to backtrack in the token stream in order to detect identifier construction using macros. Something like the following requires that 'prefix' be lexed again after macro substitution in order to detect if the string from suffix and 'prefix' will be merged into one identifier. define suffix(name) name prefix`suffix We use this often in RTL for bus port lists. Even though the spec seems to explicitly disallow this, Modelsim and DC will accept it. Lexing twice solves this case easily but now the tokens point to a non-existent source. On Mon, Apr 4, 2011 at 8:59 PM, Martin d'Anjou poin...@magma.ca wrote: Hi, Thanks to both of you for sharing your approaches. Right now I am pondering how to alter the sequence of tokens before they hit the parser. Intuitively I want to have three processing units (lexer, pre-processor, parser) connected together through io pipes of tokens (e.g. token fifos), but this is not how ANTLR was architected (it's how I would have done it in hardware though!). Martin On 11-04-04 09:25 AM, Sam Harwell wrote: I used a hand-crafted implementation of TokenSource between the lexer and parser. In the preprocessor, whenever I manipulated a token I used a new token class derived from CommonToken (call it SubstitutedToken) which contained a linked list leading from the effective position in the stream (stored in CommonToken) all the way back to the original location (file and position) of the token definition. When a CommonToken substitution occurs, the linked list has one node containing the original source position where defined. Whenever a SubstitutedToken substitution occurs, a new node for the token's previous effective position is added to the linked list and that new head pointer is stored in the new token. `define x 3 `define y `x `y In this case, token `y is eventually replaced with a SubstitutedToken which appears at (line 2, column 1, length 1, text 3) containing the following linked list: Line 3, column 1, length 2 (list head, the location where `y was substituted with `x) Line 2, column 11, length 2 (the location where `x was substituted with '3') Line 1, column 11, length 1 (the actual source location where the token '3' is defined) This list allows true relative ordering of all tokens in the processed source: when two tokens appear to be at the same location in the preprocessed stream, you simply compare the positions of the first node in the position list. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of A Z Sent: Monday, April 04, 2011 12:13 AM To: Martin d'Anjou Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Q: how to incorporate a preprocessor in the flow? Hi Martin, I just completed an SV preprocessor which can parse UVM 1.0 successfully. After 2 revisions I settled on a completely separate preprocessor(lexer and parser). As you saw, you need to tokenize the macro_text in order to easily support macros with arguments and detect the three escaped tokens `, `\` and ``. I'm not sure how well a lexer only approach could handle cases where a macro substitution can merge text with a previously lexed token. The separate approach still has flaws, such as good error reporting. Of course I could be missing an obvious easy solution. On Sun, Apr 3, 2011 at 9:51 PM, Martin d'Anjoupoin...@magma.ca wrote: Hello, I am trying to find a way to incorporate a preprocessor in the ANTLR flow. I thought of doing this before the lexer, but I need to tokenize the incoming char stream for macro substitution to be easy. I thought of doing it between the lexer and the parser, and replace the preprocessor tokens with their expansion before feeding the token stream to the parser, so I guess I would end up using something like the TokenRewriteStream??? Can someone steer me in the right direction please? Or should I be using lexer rule actions? In which case, any example on how to access the token stream of the replacement token list
[il-antlr-interest: 32109] Re: [antlr-interest] [CSharp3] rule visibility in composite grammars
Hi Ranco, I made the following changes so far: * An imported grammar is always generated as an internal class instead of a public class. * Rules within the imported grammar are always declared public since they are only called by the root grammar. How does that sound? Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ranco Marcus Sent: Wednesday, March 23, 2011 6:40 AM To: antlr-interest@antlr.org Subject: [antlr-interest] [CSharp3] rule visibility in composite grammars Hi all/Sam, While using the CSharp3 target on a composite grammar, I came across a few issues regarding rule visibility. If no modifier (private, protected, public, ...) is specified for a rule, the Java target generates public methods (since it is not implemented IIRC) and the CSharp3 target generates private methods. I like the idea to explicitly specify which rules are callable (by making them public). However, for composite grammars the generated parser contains delegate rules for every rule of the imported grammar, even for rules in the imported grammar that aren't public (causing compile errors). This can solved this adding a public modifier to every rule of the imported grammar, but I was wondering if there's a more convenient way of doing this. Furthermore, the methods for the delegate rules in the composite grammars are all public, regardless of the visibility of the rule in the imported grammar. IMHO, it would be better to make them private and use the composite grammar to explicitly specify the rules that are externally visible. What are your thoughts on this? Btw, the CSharp3 templates are also included in the original Java tool. Does that mean that we don't need the .NET version of the tool anymore? Best regards, Ranco Marcus Epirion Knowledge Solutions B.V. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32085] Re: [antlr-interest] Examples of semantic predicates with ...
Hi James,
In this case, I'm dealing with semantic predicates of the form {...}? and
gated semantic predicates of the form {...}?=. The ones you're using in
this case are syntactic predicates, which aren't processed by the Boolean
logic algorithm I've been working on recently.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of James Ladd
Sent: Tuesday, April 05, 2011 12:09 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] Examples of semantic predicates with ...
Hi Sam,
When I dealt with semantic predicates I put the unique prefix part of the
rule into a predicate.
I put the predicate on those rules that ANTLR said were ambiguous.
For example:
| (binarySelector) = binarySelector variableName (MINUS MINUS) =
| specialBinarySelector variableName
I found out most of the information on predicates by googling ANTLR
Semantic Predicates
Rgs, James.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32088] Re: [antlr-interest] Which to use StringTempates version 3 or 4?
ST4 is *tremendously* faster than ST3 and includes the most useful debugger for any template engine I've seen to date. For a new application, it's not even a question IMO - definitely ST4. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of The Researcher Sent: Tuesday, April 05, 2011 9:26 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Which to use StringTempates version 3 or 4? I have the lexing, parsing and tree rewriting completed and will now be doing a proof of concept for the next phase to see if StringTemplates will work as expected. This will be my first use of StringTemplates. Since ST4 is so new and is not documented in the books, should I start with ST3 or jump into ST4? Thanks, Eric List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32096] Re: [antlr-interest] AST cardinality from called rules
The AST operators are the most efficient manner: localVariableDeclaration : LOCAL^ variableDeclarationList ; variableDeclarationList : variableDcl ( ','! variableDcl )* ; Or use rewrites: localVariableDeclaration : LOCAL variableDeclarationList - ^(LOCAL variableDeclarationList) ; variableDeclarationList : variableDcl ( ',' variableDcl )* - variableDcl+ ; -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Graham Mer Sent: Tuesday, April 05, 2011 4:40 PM To: antlr-interest@antlr.org Subject: [antlr-interest] AST cardinality from called rules Hello list, The following rule generates the AST that I want: localVariableDeclaration : LOCAL variableDcl ( ',' variableDcl )* - ^(LOCAL variableDcl)+ ; But how do I generate an equivalent AST if the variableDcl list is not in the localVariableDeclaration rule, as in the following fragment? localVariableDeclaration : LOCAL variableDeclarationList ; variableDeclarationList : variableDcl ( ',' variableDcl )* ; I want to produce a set of ^(LOCAL variableDcl) subtrees, but I want to reuse the variableDeclarationList from other rules, so I'd rather not inline it. Thanks! List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32067] [antlr-interest] Examples of semantic predicates with extremely long generated code
Hi everyone, For certain cases of semantic predicates, ANTLR generates some *extremely* long, complicated, and redundant conditionals in the output code. I've been working on resolving this issue, and I'm looking for some examples of these grammars to make sure my updates properly reduce these conditionals. Does anyone have a sample they can send me? Thanks! Sam List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32069] Re: [antlr-interest] Examples of semantic predicates with extremely long generated code
For the example at the bottom of ANTLR-245, I get the following condition in args, element, and element2 (respectively): !true)||(false !((false)) !((true)) The tool doesn't have target language analysis, so true and false here might as well be X and Y. I'll work on getting this into the Java version after some more testing. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Terence Parr Sent: Monday, April 04, 2011 12:02 PM To: antlr-interest Interest Subject: Re: [antlr-interest] Examples of semantic predicates with extremely long generated code Looks like I have three different bugs entered http://www.antlr.org/jira/browse/ANTLR-393 http://www.antlr.org/jira/browse/ANTLR-430 http://www.antlr.org/jira/browse/ANTLR-245 I recall that at least one of them had a small example. T On Apr 4, 2011, at 9:58 AM, Sam Harwell wrote: Hi everyone, For certain cases of semantic predicates, ANTLR generates some *extremely* long, complicated, and redundant conditionals in the output code. I've been working on resolving this issue, and I'm looking for some examples of these grammars to make sure my updates properly reduce these conditionals. Does anyone have a sample they can send me? Thanks! Sam List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32073] Re: [antlr-interest] Examples of semantic predicates with extremely long generated code
Hi Loring, Yes, all my changes are in SemanticContext.cs (which will be SemanticContext.java in the Java version). Sam -Original Message- From: Loring Craymer [mailto:lgcray...@yahoo.com] Sent: Monday, April 04, 2011 12:50 PM To: Sam Harwell; Terence Parr; antlr-interest Interest Subject: Re: [antlr-interest] Examples of semantic predicates with extremely long generated code Sam-- Are you working on the SemanticContext classes for this? That seems to be where the problem originates--no effort is made to refactor generated trees. --Loring - Original Message From: Sam Harwell sharw...@pixelminegames.com To: Terence Parr pa...@cs.usfca.edu; antlr-interest Interest antlr-interest@antlr.org Sent: Mon, April 4, 2011 10:23:40 AM Subject: Re: [antlr-interest] Examples of semantic predicates with extremely long generated code For the example at the bottom of ANTLR-245, I get the following condition in args, element, and element2 (respectively): !true)||(false !((false)) !((true)) The tool doesn't have target language analysis, so true and false here might as well be X and Y. I'll work on getting this into the Java version after some more testing. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Terence Parr Sent: Monday, April 04, 2011 12:02 PM To: antlr-interest Interest Subject: Re: [antlr-interest] Examples of semantic predicates with extremely long generated code Looks like I have three different bugs entered http://www.antlr.org/jira/browse/ANTLR-393 http://www.antlr.org/jira/browse/ANTLR-430 http://www.antlr.org/jira/browse/ANTLR-245 I recall that at least one of them had a small example. T On Apr 4, 2011, at 9:58 AM, Sam Harwell wrote: Hi everyone, For certain cases of semantic predicates, ANTLR generates some *extremely* long, complicated, and redundant conditionals in the output code. I've been working on resolving this issue, and I'm looking for some examples of these grammars to make sure my updates properly reduce these conditionals. Does anyone have a sample they can send me? Thanks! Sam List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-addre ss List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32056] Re: [antlr-interest] [CSharp3] TraceIn/TraceOut and ANTLR_TRACE conditional
Hi Peter,
The TraceIn/TraceOut methods in Parser and BaseRecognizer have different
signatures (2 parameters vs. 3). The code generator emits calls to the one
in Parser, which is supposed to forward the calls to BaseRecognizer. From
what I can tell from the code generation templates, the methods in
BaseRecognizer should not have the ConditionalAttribute applied to them.
I'll remove the attribute for the next build, which will restore the
expected behavior to the methods in Parser and Lexer.
Thanks for the report. :)
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Peter Kooiman
Sent: Saturday, April 02, 2011 11:42 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] [CSharp3] TraceIn/TraceOut and ANTLR_TRACE
conditional
After struggling a bit to get tracing working in my parser by defining
ANTLR_TRACE and getting nowhere, I realized the following, which may or may
not be intended:
Both the Parser and Lexer classes inherit from BaseRecognizer and override
(well, hide) the BaseRecognizer TraceIn and TraceOut methods. These
overrides call base.Traceout / base.TraceIn, BUT, since during library
compilation ANTLR_TRACE is not defined, these base method calls become
no-ops. The result is, that while BaseRecognizer has usable TraceIn and
TraceOut methods, the TraceIn and TraceOut methods of both Parser and Lexer
become essentially no-ops.
This is the MSIL for Parser::TraceIn in Antlr3.Runtime.dll:
.method public hidebysig newslot virtual instance void TraceIn(string
ruleName,
int32 ruleIndex) cil managed
{
.custom instance void
[mscorlib]System.Diagnostics.ConditionalAttribute::.ctor(string) = ( 01 00
0B 41 4E 54 4C 52 5F 54 52 41 43 45 00 00 ) // ...ANTLR_TRACE..
// Code size 1 (0x1)
.maxstack 8
IL_: ret
} // end of method Parser::TraceIn
Fixed easily enough by overriding TraceIn/TraceOut in my parser, but a bit
of a pain to figure out, so I thought I'd at least document it here.
Regards
Peter
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32037] [antlr-interest] Problem and commentary on semantic predicates and prediction
I'm trying to use semantic predicates in a lexer. I'll try to explain
everything as I walk through the things I tried. I started with this:
TEXT
: ( ~('}'|''|'\n'|'\r'|'\\'|''|'')
| '\\' .
| {AnonymousTemplateLevel==0}? '}'
| {Outermost!=String}? ''
| {Outermost!=BigString || input.LA(2)!=''}? ''
)+
;
The TEXT token continues to consume characters until the end of the TEXT
section is complete. Depending on context (stored as fields in the lexer),
the text section could end with '}', '', or ''. When I used the above
form of the rule, the prediction section of mTokens() never evaluated my
predicates, so when the next character of the input stream was any of '}',
'', or '', it would predict the TEXT rule. Here's a shortened form of the
prediction. For simplicity, I'll use (x in SetA) as syntax for x in
~('}'|''|'\n'|'\r'|''|''), which is a character from the first alt of
TEXT or the '\' character of the second alt. I'll use SetB for
~(''|'\n'|'\r'), which is any possible character in TEXT without
considering the predicates.
if ((LA5_0 in SetB)) {
alt5 = 5; // TEXT
}
I realized shortly after that no other rule in the lexer could start with
these characters, so by process of elimination I supposed this kind of
prediction is a reasonable result. I tried to correct the situation by
adding the following rule after TEXT:
ANYCHAR : . ;
This changed the prediction in mTokens, but certainly didn't fix it.
if ((LA5_0=='\')) {
int LA5_6 = input.LA(2);
if (LA5_6 in SetB) {
alt5 = 5;
} else if (((Outermost!=String))) {
alt5 = 5;
} else if ((true)) {
alt5 = 6;
}
}
Next, looking at the ACyclicDFACodeGenerator in the tool, I noticed that
predicates are only emitted in the prediction phase if they are gated
(assigned to the predicates attribute of the dfaEdge template in
WalkFixedDFAGeneratingStateMachine). So I added gates to my predicates. In
this case, the prediction is correct, but the Boolean expression is
expressed as ((A B) || B), which reduces to simply (B). In this case, the
expense of computing B (a semantic predicate) has no bearing on the result
because there is no way to avoid evaluating it.
if ((LA5_0=='\')) {
int LA5_6 = input.LA(2);
if ((LA5_6 in SetB) ((Outermost!=String))) {
alt5 = 5;
} else if (((Outermost!=String))) {
alt5 = 5;
} else if ((true)) {
alt5 = 6;
}
}
I originally didn't see the correctness of this test, so I went on to try
removing the ANYCHAR rule while keeping the gates on the predicates. The
results are interesting, so I'll note them here. The result in mTokens
looked like this:
... } else if (LA5_0 in SetA) {
alt5 = 5;
} else if ((LA5_0=='}') ((AnonymousTemplateLevel==0))) {
alt5 = 5;
} else if ((LA5_0=='\') ((Outermost!=String))) {
alt5 = 5;
} else if ((LA5_0=='') ((Outermost!=BigString || input.LA(2)!=''))) {
alt5 = 5;
} else ...
In this case I observed the following:
1. The LL(1) prediction that now appears is faster.
2. This lexer is used as a syntax highlighting component, where removing the
ANYCHAR rule can lead to substantial performance degradation. However, since
the Tokens rule is implicitly created by the grammar compiler, I have no way
to specify that the decision on TEXT can/should be treated as LL(1).
3. The check (LA5_0 in SetA) is overly restrictive because an conditions
check for (LA_5 in (''|'\n'|'\r')). Relaxing the test reduces the number of
comparisons in the false case (say (LA5_0=='{')) from 14 to 8.
4. The test (LA5_0 in SetB) expands to the following. Edit here: I meant to
show this for SetA, but since I'll accidentally used SetB, we'll go with
that.
((LA5_5='\u' LA5_5='\t')||(LA5_5='\u000B'
LA5_5='\f')||(LA5_5='\u000E' LA5_5=';')||(LA5_5='='
LA5_5='\u'))
A simple transform (moving a few parenthesis) of this already ordered
expression gives:
((LA5_5 = '\u' (LA5_5 = '\t' || (LA5_5 = '\u000B' (LA5_5 =
'\f' || (LA5_5 = '\u000E' (LA5_5 = ';' || (LA5_5 = '=' LA5_5 =
'\u'
Which reduces the number of comparisons from 5 to 3 for eliminating '\n', 6
to 5 for eliminating '\r', and remains at 7 for eliminating '}'. For sets
with a very large number of ranges such as the ID start character in the
Java grammar, the time savings for this change could be tremendous
considering nearly all identifiers start with a character in the range
0..127 which is tested in the early portion of the expression.
Thanks,
Sam
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this
[il-antlr-interest: 32038] Re: [antlr-interest] [stringtemplate-interest] Release of StringTemplate 4.0
I already created one (attached) if you want to use it as a start point. :) It uses build-helper-maven-plugin to avoid having to restructure the directories in P4. Sam -Original Message- From: stringtemplate-interest-boun...@antlr.org [mailto:stringtemplate-interest-boun...@antlr.org] On Behalf Of Jim Idle Sent: Thursday, March 31, 2011 8:49 AM To: stringtemplate-interest List Cc: antlr-interest Interest Subject: Re: [stringtemplate-interest] [antlr-interest] Release of StringTemplate 4.0 Shall I create the Maven build or are you throwing Maven out? ;-) Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Terence Parr Sent: Sunday, March 27, 2011 12:55 PM To: stringtemplate-interest List Cc: antlr-interest Interest Subject: [antlr-interest] Release of StringTemplate 4.0 Get it while it's hot. 4.0 is a complete rewrite as a bytecode interpreter. Is much cleaner and faster than 3.0. enjoy. http://www.stringtemplate.org/download.html http://www.antlr.org/wiki/display/ST4/StringTemplate+4+Documentation http://www.antlr.org/wiki/display/ST4/Differences+between+v3+and+v4 The GUI Visualizer / inspector is very useful; check it out: http://www.antlr.org/wiki/display/ST4/StringTemplate+Inspector+GUI Ports to C#, Python, Objective-C on the way thanks to Sam Harwell, Benjamin Niemann, and Alan Condit. A big thanks to Udo Borkowski for his incredible bug catching abilities and his suggestions! Terence ANTLR project supreme dictator for life List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address ___ stringtemplate-interest mailing list stringtemplate-inter...@antlr.org http://www.antlr.org/mailman/listinfo/stringtemplate-interest List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address stringtemplate.config Description: XML document -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32040] Re: [antlr-interest] Problem and commentary on semantic predicates and prediction
By reordering on binary search basis, the number of comparisons becomes 3 for '\n', 3 for '\r', and 3 for ''. In other words, the way comparisons are expressed even for a set with only 4 sub-ranges results in 7 comparisons for characters where only 3 are actually needed. This is O(log n) instead of O(n), so as the interval set gets more ranges the discrepancy quickly grows - for 8 sub-ranges used in testing SetA from before, 14 comparisons are used where only 4 are needed. (LA5_0 = '\f' (LA5_0 = '\u000B' || (LA5_0 = '\t' LA5_0 = '\u'))) || ((LA5_0 = ';' LA5_0 = '\u000E') || (LA5_0 = '=' LA5_0 = '\u')) Sam -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: Thursday, March 31, 2011 10:15 AM To: antlr-interest@antlr.org Subject: Problem and commentary on semantic predicates and prediction 4. The test (LA5_0 in SetB) expands to the following. Edit here: I meant to show this for SetA, but since I'll accidentally used SetB, we'll go with that. ((LA5_5='\u' LA5_5='\t')||(LA5_5='\u000B' LA5_5='\f')||(LA5_5='\u000E' LA5_5=';')||(LA5_5='=' LA5_5='\u')) A simple transform (moving a few parenthesis) of this already ordered expression gives: ((LA5_5 = '\u' (LA5_5 = '\t' || (LA5_5 = '\u000B' (LA5_5 = '\f' || (LA5_5 = '\u000E' (LA5_5 = ';' || (LA5_5 = '=' LA5_5 = '\u' Which reduces the number of comparisons from 5 to 3 for eliminating '\n', 6 to 5 for eliminating '\r', and remains at 7 for eliminating ''. For sets with a very large number of ranges such as the ID start character in the Java grammar, the time savings for this change could be tremendous considering nearly all identifiers start with a character in the range 0..127 which is tested in the early portion of the expression. Thanks, Sam List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32023] Re: [antlr-interest] antlr v4 wish list
Hi Martin, For lexers you can implement CharStream (ICharStream in the C# target), and for parsers you can implement TokenStream (ITokenStream in C# target). I've successfully used the former for case-insensitive lexing, and the latter for a preprocessor. ANTLRReaderStream is one of the included implementations of CharStream, and at least for .NET offers support for io pipes (which appear in the .NET framework as either TextReader (Console.In), or Stream objects such as FileStream which can be passed as new StreamReader(file)). Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Martin d'Anjou Sent: Tuesday, March 29, 2011 11:45 PM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] antlr v4 wish list Also, how about support for io pipes to decouple the lexer from the parser, such that each could be in its own thread? Also, the input to the lexer could be an io pipe. Might be easier to place a pre-processor in front of the lexer this way, and also to handle gigantic files. Martin List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32027] Re: [antlr-interest] antlr v4 wish list
Hi Matt, I agree except that it should hold the length of the token in the input stream instead of the end (where end=start+length-1). This allows insertion of zero-length tokens such as the implied ; tokens in Go (language by Google) without storing confusing positions like (start=3, end=2) - instead you get (start=3, length=0). Length and End are technically equivalent, but length leaves no questions about endpoints (inclusive/exclusive, and what to do with zero-length tokens). In my first comment, I was simply stating that you only need to store 2 values (start/length) instead of 4 (start/length/line/column), because the latter two can be efficiently derived from the former. In ANTLR v4 the line map I mentioned will likely be included in the base lexer implementation, so it will be efficient (space and time) without becoming a burden. Sam -Original Message- From: Matt Fowles [mailto:matt.fow...@gmail.com] Sent: Wednesday, March 30, 2011 8:57 AM To: Sam Harwell Cc: Martin d'Anjou; antlr-interest@antlr.org Subject: Re: [antlr-interest] antlr v4 wish list Sam~ A token needs to know both start and end position. Especially when you add in the restriction that *synthetic* tokens should respond with the positions for the entire rule that created them (if they weren't based on another token). Basically, you need Tree and Token to always be able to provide locations in the original stream (even if those locations are best guess) regardless of how many tree transformations have taken place. Whether it internally uses a shared array of line offsets or stores duplicates in every token, I don't care, but pushing all of that onto every language implementer is not a good trade off. Matt On Tue, Mar 29, 2011 at 11:29 PM, Sam Harwell sharw...@pixelminegames.com wrote: Hi Martin, Replying to the individual points: 1. A token only needs to know the start position in the input stream and the length. Considering a file may easily have hundreds of thousands of tokens, it's very important to not add any information to the token that can be efficiently derived in another manner, especially if that information is infrequently used by applications. For example, the line/column information can be efficiently derived if the lexer maintains an internal array of line offsets (index 0 contains 0, the start position of line 0; index 1 contains the offset to the start of line 1; etc...). 3. The current notation is pretty simple once you see it. Also, it's well documented in the books. 4. With proper integration into the build system, generated files aren't checked into source control or distributed. The ANTLR project itself generates V2 and V3 grammars, and my .NET projects generate V3 grammars (using my C# port of the Tool) at build time, so the generated files never take up space in source control. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Martin d'Anjou Sent: Tuesday, March 29, 2011 9:33 PM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] antlr v4 wish list Hello, My suggestions, for what it's worth: 1) In the Runtime section: * Tokens and Trees should both know their start/stop line, start/stop char position to make IDEs easier. Not only IDEs, but for also for debugging on the command line in a terminal. The file name is also needed. 2) Lexer debug enhancement: Option on the lexer constructor to have the lexer print some debug info: token type by name, token value, filename, line and char position, without having to replace antlr's built-in classes. 3) General: I have spent many hours on a ridiculous little problem: the grammar declaration statement! So I suggest enforcing the grammar type in the grammar declaration: parser grammar MyGrammar; lexer grammar MyGrammar; mixed grammar MyGrammar; // lexer and parser grammar tree grammar MyGrammar; 4) Gigantic source files, as described here: http://v2kparse.blogspot.com/2008/06/first-pass-uploaded-to-sourceforc e.html Maybe this has been solved already? Regards, Martin d'Anjou List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32016] Re: [antlr-interest] antlr v4 wish list
Hi Martin, Replying to the individual points: 1. A token only needs to know the start position in the input stream and the length. Considering a file may easily have hundreds of thousands of tokens, it's very important to not add any information to the token that can be efficiently derived in another manner, especially if that information is infrequently used by applications. For example, the line/column information can be efficiently derived if the lexer maintains an internal array of line offsets (index 0 contains 0, the start position of line 0; index 1 contains the offset to the start of line 1; etc...). 3. The current notation is pretty simple once you see it. Also, it's well documented in the books. 4. With proper integration into the build system, generated files aren't checked into source control or distributed. The ANTLR project itself generates V2 and V3 grammars, and my .NET projects generate V3 grammars (using my C# port of the Tool) at build time, so the generated files never take up space in source control. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Martin d'Anjou Sent: Tuesday, March 29, 2011 9:33 PM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] antlr v4 wish list Hello, My suggestions, for what it's worth: 1) In the Runtime section: * Tokens and Trees should both know their start/stop line, start/stop char position to make IDEs easier. Not only IDEs, but for also for debugging on the command line in a terminal. The file name is also needed. 2) Lexer debug enhancement: Option on the lexer constructor to have the lexer print some debug info: token type by name, token value, filename, line and char position, without having to replace antlr's built-in classes. 3) General: I have spent many hours on a ridiculous little problem: the grammar declaration statement! So I suggest enforcing the grammar type in the grammar declaration: parser grammar MyGrammar; lexer grammar MyGrammar; mixed grammar MyGrammar; // lexer and parser grammar tree grammar MyGrammar; 4) Gigantic source files, as described here: http://v2kparse.blogspot.com/2008/06/first-pass-uploaded-to-sourceforce.html Maybe this has been solved already? Regards, Martin d'Anjou List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31939] Re: [antlr-interest] [CSharp3] @namespace and composite grammars
Hi Ranco,
As mentioned in the CSharp3 documentation, you need to use both
@lexer::namespace{...} and @parser::namespace{...} in a composite grammar.
If you use the templates included in my Visual Studio extension, then these
are automatically inserted for you (including using the correct namespace
based on the location in the project where you placed them).
I'll check into the namespace issue when it comes to imported grammars. The
expected functionality is this should work once you have
@lexer::namespace{...} and @parser::namespace{...} in the composite grammar
, plus @namespace{...} in each of P and L.
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ranco Marcus
Sent: Thursday, March 24, 2011 5:55 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] [CSharp3] @namespace and composite grammars
To maximize reuse of our grammars we generally create separate lexer and
parser grammars which are target agnostic (i.e. contain no target specific
code). We then combine them into a composite grammar in which the target
language and other implementation specific details are specified
(header,members,namespace,etc.). This way, grammars can be used for multiple
targets and we have maximum freedom to combine multiple lexers/parsers into
larger ones.
If a composite grammar C imports a parser grammar P and a lexer grammar L,
the tool generates CParser, CLexer, C_P and C_L.
Adding @namespace { X } to the composite grammar should IMHO put all four
recognizers in namespace X. Currently, at least with the CSharp3 target,
only CParser is put into the specified namespace.
Specifying @lexer::namespace { X } to the composite grammar causes only
the outermost lexer CLexer to be added to the specified namespace, not the
imported lexer C_L. The same holds for @parser::namespace.
Btw, if I'm not mistaken, there's no way to put recognizers in packages with
the Java target. Is that correct?
Best regards,
Ranco Marcus
Epirion Knowledge Solutions B.V.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31941] Re: [antlr-interest] [CSharp3] @namespace and composite grammars
Hi Kevin,
The plugin information is included in the Visual Studio and the ANTLR C#
Target documentation referenced on the ANTLR wiki:
http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases
Thanks,
Sam
From: Kevin Cherry [mailto:kche...@tigers.lsu.edu]
Sent: Thursday, March 24, 2011 9:12 AM
To: Sam Harwell
Cc: Ranco Marcus; antlr-interest@antlr.org
Subject: Re: [antlr-interest] [CSharp3] @namespace and composite grammars
@Sam, what is the name of the Visual Studio extension you are talking about?
I tried to search for anything with ANTLR in it and it couldn't find
anything.
On Thu, Mar 24, 2011 at 8:50 AM, Sam Harwell sharw...@pixelminegames.com
wrote:
Hi Ranco,
As mentioned in the CSharp3 documentation, you need to use both
@lexer::namespace{...} and @parser::namespace{...} in a composite grammar.
If you use the templates included in my Visual Studio extension, then these
are automatically inserted for you (including using the correct namespace
based on the location in the project where you placed them).
I'll check into the namespace issue when it comes to imported grammars. The
expected functionality is this should work once you have
@lexer::namespace{...} and @parser::namespace{...} in the composite grammar
, plus @namespace{...} in each of P and L.
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ranco Marcus
Sent: Thursday, March 24, 2011 5:55 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] [CSharp3] @namespace and composite grammars
To maximize reuse of our grammars we generally create separate lexer and
parser grammars which are target agnostic (i.e. contain no target specific
code). We then combine them into a composite grammar in which the target
language and other implementation specific details are specified
(header,members,namespace,etc.). This way, grammars can be used for multiple
targets and we have maximum freedom to combine multiple lexers/parsers into
larger ones.
If a composite grammar C imports a parser grammar P and a lexer grammar L,
the tool generates CParser, CLexer, C_P and C_L.
Adding @namespace { X } to the composite grammar should IMHO put all four
recognizers in namespace X. Currently, at least with the CSharp3 target,
only CParser is put into the specified namespace.
Specifying @lexer::namespace { X } to the composite grammar causes only
the outermost lexer CLexer to be added to the specified namespace, not the
imported lexer C_L. The same holds for @parser::namespace.
Btw, if I'm not mistaken, there's no way to put recognizers in packages with
the Java target. Is that correct?
Best regards,
Ranco Marcus
Epirion Knowledge Solutions B.V.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31910] Re: [antlr-interest] [CSharp3] rule visibility in composite grammars
Hi Ranco, I'll add a simple delegate grammar example to the tests I use here and find a way to make it work. :) If you are using Visual Studio (or MSBuild in particular, which is the standard VS build system), then you'll definitely want to use the C# port of the tool. I highly recommend this route because it's both easy to work with and doesn't prod you to check your generated grammar files into source control. While the following documentation makes specific references to Visual Studio 2010, everything except section 1.1 (the IDE language support package) applies equally to Visual Studio 2008. http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases Thanks, Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ranco Marcus Sent: Wednesday, March 23, 2011 6:40 AM To: antlr-interest@antlr.org Subject: [antlr-interest] [CSharp3] rule visibility in composite grammars Hi all/Sam, While using the CSharp3 target on a composite grammar, I came across a few issues regarding rule visibility. If no modifier (private, protected, public, ...) is specified for a rule, the Java target generates public methods (since it is not implemented IIRC) and the CSharp3 target generates private methods. I like the idea to explicitly specify which rules are callable (by making them public). However, for composite grammars the generated parser contains delegate rules for every rule of the imported grammar, even for rules in the imported grammar that aren't public (causing compile errors). This can solved this adding a public modifier to every rule of the imported grammar, but I was wondering if there's a more convenient way of doing this. Furthermore, the methods for the delegate rules in the composite grammars are all public, regardless of the visibility of the rule in the imported grammar. IMHO, it would be better to make them private and use the composite grammar to explicitly specify the rules that are externally visible. What are your thoughts on this? Btw, the CSharp3 templates are also included in the original Java tool. Does that mean that we don't need the .NET version of the tool anymore? Best regards, Ranco Marcus Epirion Knowledge Solutions B.V. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31849] Re: [antlr-interest] @scopeinit, can find the post but nothing else.
Hi Eric, In the C# 3 documentation, section 4.4 discusses partial methods called when a scope object is created. http://www.antlr.org/download/CSharp3.pdf Thanks, Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of The Researcher Sent: Thursday, March 03, 2011 2:49 PM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] @scopeinit, can find the post but nothing else. This appears to be on the To Implement list constructors or init for global scopes http://www.antlr.org/wiki/display/ANTLR3/constructors+or+init+for+global+sco pes Eric On Wed, Mar 2, 2011 at 12:26 PM, The Researcher researcher0...@gmail.comwrote: Sam, I saw your post about @scopeinit which I take was related to Trouble with syntactic predicates that reference semantic predicates that rely on a current dynamic attribute scope. Did @scopeinit make it into a released C# version? If so could you point me to how it is used? I can get by without it, but my current solution is a crutch at best. Thanks Eric. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31850] Re: [antlr-interest] C# tree parser memoize issue?
Hi David,
Is there any way you can send me the full grammar you're using (you could
send it to just me if you don't want to send it to the public list)? I don't
use memoization in any of my grammars, so I don't have a way to test this
feature extensively.
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of David Daeschler
Sent: Thursday, March 03, 2011 1:45 PM
To: antlr-interest@antlr.org
Subject: [antlr-interest] C# tree parser memoize issue?
I have a grammar that contains a 3d vector type.
Parsing the grammar for my test case yields the following AST:
http://graph.gafol.net/dZxqGkwww
http://graph.gafol.net/dZxqGkwwwDuring the code generation step I am using
a tree grammar with backtracking and memoization turned on.
With memoization the following generates an error:
float f;
vector v = 1.0,1.0,1.0 * f,f,f;
^Z
Gen.g: node from line 2:28 mismatched tree node: f expecting FLOAT_LITERAL
Turning off memoization fixes the issue.
The grammar fragments involved are (trying to cut down as much as I can):
expression
@init {MyAst t = (MyAst )input.LT(1);}
@after { $expression.st = DoPromotion(t, $expression.st); }
: ^(EXPR expr) - {$expr.st}
;
expr
: multiplicativeExpression- {$multiplicativeExpression.st} ;
multiplicativeExpression
: mult - {$mult.st}
| primary - {$primary.st}
;
mult
: ^(op='*' l=multiplicativeExpression r=multiplicativeExpression)
-
mul(subtemplate={TemplateMapping.Multiplication[$l.start.evalType.TypeIndex,
$r.start.evalType.TypeIndex]}, lexpr={$l.st}, rexpr={$r.st}) ; primary @init
{MyAst t = (MyAst)input.LT(1);} @after { $primary.st = DoPromotion(t,
$primary.st); }
| vecConst - {$vecConst.st}
| vecLiteral - {$vecLiteral.st}
| expression - {$expression.st}
;
vecConst
: ^(VECTOR_LITERAL x=FLOAT_LITERAL y=FLOAT_LITERAL z=FLOAT_LITERAL) -
vconst(x={$x.text}, y={$y.text}, z={$z.text}) ;
vecLiteral
: ^(VECTOR_LITERAL x=expr y=expr z=expr) - buildvec(x={$x.st}, y={$y.st},
z={$z.st}) ;
Sorry for the verbosity of this post, and thank you ahead of time for any
help.
--
David Daeschler
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31792] Re: [antlr-interest] CSharp3 Composite error
Hi Andrey, Thanks for the info. I'll see to getting it in for the next release. :) Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Andrey Sirotkin Sent: Friday, March 11, 2011 8:13 AM To: antlr-interest@antlr.org Subject: [antlr-interest] CSharp3 Composite error Hello! ... Move the parserCtorBody() after grammar.directDelegates and now it works correctly. Kind regards, Andrey Sirotkin List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31733] Re: [antlr-interest] [SPAM] [SPAM] Re: C# Target: Problems with Mono
The CSharp2 target is currently broken, but it's in the process of getting fixed. The maintainer has been very busy, and it's been especially difficult given the extra features I've recently added to the CSharp3 target which aren't supported in C# 2. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Felix Natter Sent: Saturday, March 05, 2011 9:25 AM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] [SPAM] [SPAM] Re: C# Target: Problems with Mono Olivier Lefevre lefev...@yahoo.com writes: On 3/1/2011 10:00 PM, Felix Natter wrote: I am trying to generate C# code from my grammar for use with Mono2 (v1.9). ** (Antlr3.exe:4230): WARNING **: Missing method System.Reflection.Emit.DynamicMethod::.ctor(string,Type,Type[]) 1) The Mono team is quite candid about what is missing: http://www.go-mono.com/status/ (System.Reflection can be found under mscorlib) 2) Mono 2.1.9 is ancient: the long-term and current stable versions are 2.6.7 and Thanks for the hint. Yesterday I upgraded to Mono 2.6.7 (backport from Debian squeeze, I haven't yet upgraded) and it seems I can now use the CSharp3 target by loading the .sln in monodevelop (but I have yet to test this more thoroughly). But I am quite sure that the CSharp2 target in ANTLR 3.3 is broken (but, as a last resort, the CSharp target in ANTLR 3.1 seems to work). 2.10.1: http://www.go-mono.com/mono-downloads/download.html 3) Has anyone run the ANTLR C# runtime through Moma, the Mono Migration Analyzer? http://mono-project.com/Moma That would be interesting. Greetings, -- Felix Natter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31670] Re: [antlr-interest] [antlr-dev] Generated code checks for backtracking even though backtracking is off
Hi Susanna,
Actions are not executed as part of a syntactic predicate. As a simplified
example, take the following:
rule : (x {Foo();}) = x;
Foo() will not be called here, instead, you should write:
rule : (x) = x {Foo();};
Sam
From: antlr-dev-boun...@antlr.org [mailto:antlr-dev-boun...@antlr.org] On
Behalf Of Susanna Siebert
Sent: Monday, February 28, 2011 2:19 PM
To: antlr-...@antlr.org
Subject: [antlr-dev] Generated code checks for backtracking even though
backtracking is off
Hi everybody,
we are trying to add custom bracematching to our language, Hop. For example,
our functions are designed in the form action myAction .. end, where we are
trying to match the action with the end. To do that we added some
semantic actions to the appropriate rules in the *.g file, that add some
information about the position of the tokens to a hash table in a matcher
module. This works fine except for the if-else-statements. These look like
so:
if_statement
:
firstif = IF root_expression THEN { symbol.add(new BlockDescriptor()); }
block { symbol.popScope(); } firstend = END
{
MatchKeyword keyword1 = new MatchKeyword();
keyword1.setMatchID(((CommonToken)$firstif).getStartIndex());
keyword1.setStartOffset(((CommonToken)$firstif).getStartIndex() -
$firstif.getLine() + 1);
keyword1.setEndOffset(((CommonToken)$firstif).getStopIndex() -
$firstif.getLine() + 2);
keyword1.setIsPrefix(true);
MatchKeyword keyword2 = new MatchKeyword();
keyword2.setMatchID(((CommonToken)$firstif).getStartIndex());
keyword2.setStartOffset(((CommonToken)$firstend).getStartIndex() -
$firstend.getLine() + 1);
keyword2.setEndOffset(((CommonToken)$firstend).getStopIndex() -
$firstend.getLine() + 2);
keyword2.setIsPrefix(false);
matcher.addMatch(keyword1, keyword2);
}
((firstelse = ELSE IF root_expression THEN { symbol.add(new
BlockDescriptor()); } block { symbol.popScope(); } secondend = END
{
MatchKeyword keyword1 = new MatchKeyword();
keyword1.setMatchID(((CommonToken)$firstelse).getStartIndex());
keyword1.setStartOffset(((CommonToken)$firstelse).getStartIndex() -
$firstelse.getLine() + 1);
keyword1.setEndOffset(((CommonToken)$firstelse).getStopIndex() -
$firstelse.getLine() + 2);
keyword1.setIsPrefix(true);
MatchKeyword keyword2 = new MatchKeyword();
keyword2.setMatchID(((CommonToken)$firstelse).getStartIndex());
keyword2.setStartOffset(((CommonToken)$secondend).getStartIndex() -
$secondend.getLine() + 1);
keyword2.setEndOffset(((CommonToken)$secondend).getStopIndex() -
$secondend.getLine() + 2);
keyword2.setIsPrefix(false);
matcher.addMatch(keyword1, keyword2);
}
) = (ELSE IF root_expression THEN block END))* //else if blocks
((secondelse = ELSE THEN { symbol.add(new BlockDescriptor()); } block {
symbol.popScope(); } thirdend = END
{
MatchKeyword keyword1 = new MatchKeyword();
keyword1.setMatchID(((CommonToken)$secondelse).getStartIndex());
keyword1.setStartOffset(((CommonToken)$secondelse).getStartIndex() -
$secondelse.getLine() + 1);
keyword1.setEndOffset(((CommonToken)$secondelse).getStopIndex() -
$secondelse.getLine() + 2);
keyword1.setIsPrefix(true);
MatchKeyword keyword2 = new MatchKeyword();
keyword2.setMatchID(((CommonToken)$secondelse).getStartIndex());
keyword2.setStartOffset(((CommonToken)$thirdend).getStartIndex() -
$thirdend.getLine() + 1);
keyword2.setEndOffset(((CommonToken)$thirdend).getStopIndex() -
$thirdend.getLine() + 2);
keyword2.setIsPrefix(false);
matcher.addMatch(keyword1, keyword2);
}
) = (ELSE THEN block END) )?
;
The problem is that when the code is generated, the semantic actions are
being surrounded by an if-statement that checks state.backtracking (if
(state.backtracking == 0) {}). For the if-then part of the rule, this check
evaluates to true but for the else-if-then and else-then it evaluates to
false because state.backtracking is 1. However, backtracking is turned off.
My questions now are:
Why is a check for state.backtracking added when code is generated even if
backtracking is turned off?
How can I make sure that the semantic actions of the else-if-then and
else-then are being executed?
I appreciate your help.
Best regards,
Susanna Siebert
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31625] Re: [antlr-interest] C# antlr-dotnet-tool-3.3.1.7705 issue with collected template list
Hi David,
Can you add the following template to the CSharp3 target's ST.stg which is
used for generating code. Let me know if it resolves the issue for you. If
this works, I'll clean it up (only overriding the necessary portion) and get
it checked in for the next release.
ruleLabelDefs() ::=
[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels,ruleDescriptor.w
ildcardTreeLabels,ruleDescriptor.wildcardTreeListLabels]
:{labelType it.label.text = default(labelType);}; separator=\n
ruleDescriptor.tokenListLabels
:{List\labelType\ list_it.label.text = null;}; separator=\n
ruleDescriptor.ruleListLabels
:{List\StringTemplate\ list_it.label.text = null;}; separator=\n
ruleDescriptor.wildcardTreeListLabels
:{List\ASTLabelType\ list_it.label.text = null;}; separator=\n
ruleDescriptor.ruleLabels:ruleLabelDef(label=it); separator=\n
ruleDescriptor.ruleListLabels:ruleLabelDef(label=it); separator=\n
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of David Daeschler
Sent: Friday, February 25, 2011 1:25 PM
To: antlr-interest@antlr.org
Subject: [antlr-interest] C# antlr-dotnet-tool-3.3.1.7705 issue with
collected template list
Hello,
I am using antlr-dotnet-tool-3.3.1.7705 and beginning work on a template to
translate from a high level language to bytecode.
One of my rules looks something like this:
script : (g+=globalVar)+
- file(globalVars={$g})
globalVar
: ^(VAR_DECL TYPE ID .?)
- vardec(type={$TYPE.text}, id={$ID.text})
;
It appears that the code being generated in this case to collect the
templates is slightly off. The type of list to hold the returned result is:
ListMyAST list_g = null;
But when the collection is done, it is expecting the list to hold a
stringtemplate:
if (list_g==null) list_g=new ListStringTemplate();
list_g.Add(g.Template);
I have confirmed my output is set correctly:
options {
language = 'CSharp3';
tokenVocab = MyVocab;
ASTLabelType = MyAST;
output = template;
}
Is there anything else I need to do?
Thank you very much for any help.
--
David Daeschler
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31628] Re: [antlr-interest] AntlrWorks FEATURE REQUEST
This is exactly when creating an IDE you (IDE developer) should always include something like the following in the parse error handler (causing it cancel out of the entire parse operation): if (errorCount N) throw new OperationCanceledException(); :) Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Douglas Godfrey Sent: Friday, February 25, 2011 1:49 PM To: antlr-interest@antlr.org Subject: [antlr-interest] AntlrWorks FEATURE REQUEST Please add an option to turn off the parse as you type syntax checking with a toolbar button to refresh the parse. When I am converting a grammar from BNF to Antlr the grammar may start out with thousands of errors. AntlrWorks GUI becomes unresponsive if the errors cause an exception in the parse as you type syntax checking. AntlrWorks GUI also becomes unresponsive if there are more than 10-20 errors even if they don't cause a parse exception. I wind up having to edit the BNF grammar in another editor and switch back and forth between my editor and AntlrWorks for each error. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31636] Re: [antlr-interest] and, finally, ternary operator works
Ternary for C++ is even stranger, though I'm not sure it's necessary to
support it at this point.
x ? a : b = c
should parse as this in C++:
(? x a (= b c))
But as this is C (which is found to be invalid at a later time since the
ternary operator cannot return an lvalue in C):
(= (? x a b) c)
However, in both C and C++ the following:
x = c ? a : b
Should parse as:
(= x (? c a b))
I've been watching for those check-ins, and I see one now! :)
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Terence Parr
Sent: Friday, February 25, 2011 7:41 PM
To: antlr-interest Interest
Subject: [antlr-interest] and, finally, ternary operator works
Wow. I'm having fun. I guess i'll bring into main line now. The beauty is
that I'm generating another grammar with very little target code needed. I
will add the following to each target:
// used for left-recursive rules
recRuleDefArg() ::= int recRuleArg()
recRuleArg() ::= _p
recRuleAltPredicate(ruleName,opPrec) ::= recRuleArg() \= opPrec
recRuleSetResultAction() ::= root_0=$ruleName_primary.tree;
That will be correct for most targets. here's output for the below test:
e_[int _p]
: e_primary {root_0=$e_primary.tree;}
(
( {_p = 6}?= '*'^ e_[7]{}
| {_p = 5}?= '+'^ e_[6]{}
| {_p = 3}?= '='assoc=right^ e_[3]{}
| {_p = 4}?= '?'assoc=right^ e ':'! e_[4]{}
)
)*
;
here's another test.
@Test public void testTernaryExpr() throws Exception {
String grammar =
grammar T;\n +
options {output=AST;}\n +
e : e '*'^ e +
| e '+'^ e +
| e '?'assoc=right^ e ':'! e +
| e '='assoc=right^ e +
| ID +
;\n +
ID : 'a'..'z'+ ;\n +
WS : (' '|'\\n') {skip();} ;\n;
String[] tests = {
a,a,
a+b, (+ a b),
a*b, (* a b),
a?b:c,(? a b c),
a=b=c,(= a (= b c)),
a?b+c:d, (? a (+ b c) d),
a?b=c:d, (? a (= b c) d),
a? b?c:d : e, (? a (? b c d) e),
a?b: c?d:e, (? a b (? c d e)),
};
runTests(grammar, tests, e);
}
ternary is pretty weird. C and Java grammars show
ConditionalExpression:
ConditionalOrExpression
ConditionalOrExpression ? Expression : ConditionalExpression
So it's right associative but middle expr acts like (expr).
Ter
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31591] Re: [antlr-interest] Antlr 3.3, CSharp3 target, StringTemplate: Invalid code generated for a template rewrite
Hi Peter,
This is fixed in the 3.3.1 release from last week. Here is a link to the
instructions which I keep updated. It includes a link to release 3.3.1 in
section 1.3.
http://www.tunnelvisionlabs.com/downloads/antlr/ANTLRCSharp3_VS2010.xps
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Peter Crowther
Sent: Thursday, February 24, 2011 6:15 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] Antlr 3.3, CSharp3 target, StringTemplate: Invalid
code generated for a template rewrite
Source fragment (it's a tree grammar, long story):
-- snip --
federation_query: ^(FEDERATIONQUERY namespace_part? warranties_part?
select_statement) - federation_query(select={select_statement}) ;
-- snip --
Generated code fragment:
-- snip --
// TEMPLATE REWRITE
// 15:89: - federation_query(select=select_statement)
{
retval.Template =
templateLib.GetInstanceOf(federation_query,
new STAttrMap().put(select, select_statement));
}
}
-- snip --
Error:
-- snip --
The type or namespace name 'STAttrMap' could not be found (are you missing a
using directive or an assembly reference?)
-- snip --
In C#, the second parameter of GetInstanceOf() is
IDictionarystring,object. Note that IDictionary does not define put(key,
value) - the .Net equivalent is Add(key, value). Is this a holdover from an
incomplete Java translation? How might I debug it?
All ideas welcome!
- Peter
--
Peter Crowther, Director, Melandra Limited
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31582] Re: [antlr-interest] ast rewrites in left-recursive rules
(Begin objective section)
Inside the following rewrite:
e : e '*' b=e - ^(...);
e refers to the first e (the one right after the ':' in the rule)
$e refers to the enclosing rule (the tree created so far)
$b refers to the second e (the one labeled 'b=')
This is never ambiguous because the following is not valid in a grammar:
e : e '*' e=e;
In particular, no label in the grammar can match the name of any rule in the
grammar, so $rulename appearing in a rewrite, where rulename is some rule in
the grammar, can only *ever* refer to an enclosing rule.
(End objective section, begin subjective section) :)
I'm sure you've noticed I've been pushing AST operators recently. :) With
the extended syntax, I've found that the vast majority (over 90%) of rules I
write use AST operators instead of rewrite syntax. Due to this, I rarely
have to worry about labels. For the rule you've written, I'd just use '*'^
and skip the rewrite. Another little known fact is the following rule:
e : NUM (LETTER^ NUM)*;
With the input 1 A 2 B 3 produces the following tree, which is a
left-associative infix operator:
(B (A 1 2) 3)
For the expression syntax you're using, AST operators remain a great
candidate as long as you can also handle right-associativity for them.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Terence Parr
Sent: Wednesday, February 23, 2011 6:37 PM
To: antlr-interest Interest
Subject: [antlr-interest] ast rewrites in left-recursive rules
So I have it working with rewrite rules now:
e : e '.' ID- ^('.' e ID)
| e '.' 'this'- ^('.' e 'this')
| '-' e - ^('-' e)
| e '*' b=e - ^('*' e $b)
| e (op='+'|op='-') b=e - ^($op e $b)
| INT - INT
| ID - ID
;
But take a look at the multiplication rule: it needs a label on the second
e. plain e is ambiguous. I decided that plain e references the left
recursive version; since it will disappear during the transformation,
putting a label on that one won't work. we have to put a label on the second
reference as you see above. this is not optimal. can anyone think of a
better way to differentiate between the left and right e references in a
single alternative? [Note that e refers to the entire tree created so far.]
Ter
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31482] Re: [antlr-interest] Compile errors with CSharp2 Target
Hi Simon,
Using the 3.3.1 build I posted last night, the only issue with the C# target
is the use of {$channel=HIDDEN;}. To match .NET naming conventions, the
hidden channel constant is Hidden, so they need to be changed to
{$channel=Hidden;}.
I updated the grammar you sent by:
1. Fixing the CSharp3 build issue (requires the 3.3.1 build I posted last
night).
2. Left factoring to remove the backtrack, memoize, and k=2 options.
3. Using the extended AST operators of the CSharp3 target to remove
unnecessary rewrite rules.
I haven't tested the result aside from making sure it compiles properly with
the CSharp3 target.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Simon
Sent: Wednesday, February 16, 2011 1:42 AM
To: antlr-interest@antlr.org
Subject: Re: [antlr-interest] Compile errors with CSharp2 Target
On Wed, Feb 16, 2011 at 10:47 AM, Johannes Luber jalu...@gmx.de wrote:
Did you read the other emails stating you may have a grammar which is
meant to work with Java? If possible pls attach the grammar to the email, so
we can check this and any other errors. I forgot which ANTLR versions
require what runtimes exactly, but you need 3.1 or 3.2 for CSharp2. CSharp3
requires ANTLR 3.3 and other assemblies than I've provided, as the targets
aren't compatible with each other yet. Not sure if those assemblies are
provided in distro or where else you can find them. Sam should know more.
Ah, thanks to everyone for pointing out the obvious - I looked more
thoroughly in the grammar and in the end did find some stray bits of Java
code which, once eliminated let it compile just fine using
CSharp2. I still have the same errors with CSharp3 (incorrect escape
sequences when generated with antlr 3.1.3, different errors with antlr
3.3). I've attached the purified Php.g grammar that succeeds in
CSharp2 but fails in CSharp3 in case it is useful - however I think I will
proceed with CSharp2 at this point.
Thanks again for the help!
Simon
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
Php.g
Description: Binary data
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31483] Re: [antlr-interest] NPE when adding @header? (ANTLRWorks 1.4.2, Mac OSX 10.6.6)
Hi Bill,
You need to place the @header block after the options{} block.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Bill Schongar
Sent: Wednesday, February 16, 2011 8:04 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] NPE when adding @header? (ANTLRWorks 1.4.2, Mac
OSX 10.6.6)
Hi folks,
I¹m just getting started with Antlr, and have a basic grammar working.
However, as soon as I add an @header declaration I get an NPE when I go to
Debug it, almost as if I hadn¹t properly terminated the @header declaration.
If I remove @header everything works fine, and I can run debug with my
sample text, get my expected output, etc.
Any thoughts on what I¹m doing wrong? System info, relevant section of the
grammar and NPE are added at the end for reference.
Thanks,
-Bill
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31497] Re: [antlr-interest] CSharp tool port
Hi Chris, I just sent an email last night about a new release. :) The links are broken on that page due to word wrap - just copy everything within the angle braces (...) and remove the newline to get the correct link. http://www.antlr.org/pipermail/antlr-interest/2011-February/040767.html Thanks, Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of chris king Sent: Wednesday, February 16, 2011 4:52 PM To: antlr-interest@antlr.org Subject: [antlr-interest] CSharp tool port Does anyone know if this is the latest version of *Sam Harwell *C# port or the Antlr tool? *http://www.280z28.org/downloads/antlr/antlr-dotnet-binaries-3.2.0.6805.7z* Thanks, Chris List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31500] Re: [antlr-interest] Context Sensitive Keyword Support?
That's actually on my list of things to discuss when I'm there. :) Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Terence Parr Sent: Wednesday, February 16, 2011 5:43 PM To: antlr-interest Interest Subject: Re: [antlr-interest] Context Sensitive Keyword Support? I also want to try a scannerless version of ANTLR. Rats guy Robert Grimm has groovy combined language / modularity feature. ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31456] Re: [antlr-interest] Compile errors with CSharp2 Target
Hi Simon,
I assume you're using the grammar from this project:
http://code.google.com/p/phpparser/
If so, you'll need to modify the grammar to work with any target other than
the Java target. In the @header and @members blocks at the top, the
contained code is written in Java and will need to be ported to C#. As a
start, the @header blocks should be:
@lexer::namespace{Kuruvila.Php.Parser}
@parser::namespace{Kuruvila.Php.Parser}
You'll also need to port the embedded action in the HereDocContents lexer
rule to C#.
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Simon
Sent: Monday, February 14, 2011 11:22 PM
To: antlr-interest@antlr.org
Subject: [antlr-interest] Compile errors with CSharp2 Target
Hello all,
I'm trying to get the CSharp2 target to work. I followed the
instructions here:
http://www.antlr.org/wiki/pages/viewpage.action?pageId=557075
except that I generated using the command line rather than using Antlrworks:
java -classpath antlr-3.1.3.jar org.antlr.Tool -o . Test.g
The test example worked just fine and produced valid CSharp code, but when I
tried a PHP grammar (which works fine when generating to Java
code) it produced a build error:
C:\Temp\PHPParser\PhpLexer.cs(132,22): error CS0722:
'Antlr.Runtime.Token': static types cannot be used as return types
I tried using various different .NET targets (2.0, 3.0, 3.5), but it didn't
help.
Can anyone give me some hints about what might be wrong here?
Cheers Thanks!
Simon
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31468] [antlr-interest] Updated C# port and CSharp3 target (C# build 3.3.1.7705)
Hello everyone, First of all, I have to make the following important note: due to a change in ANTLRCore.sti, this build of Antlr3.exe (the tool) only works properly with the included CSharp3 target. Likewise, the CSharp3 target only works properly with this version of the tool (the Java version will not work). I'll have to post the list of changes since the 3.3.0 release online later. For the distribution, there are several options here: . CSharp3Runtime http://www.tunnelvisionlabs.com/downloads/antlr/antlr-dotnet-csharp3runtime -3.3.1.7705.7z : Only includes the latest build of Antlr3.Runtime.dll. . CSharp3Boostrap http://www.tunnelvisionlabs.com/downloads/antlr/antlr-dotnet-csharp3bootstr ap-3.3.1.7705.7z : Includes all files necessary to generate code for (and run) grammars using the CSharp3 target. It also includes the MSBuild targets file to include ANTLR in the Visual Studio build process. . Tool http://www.tunnelvisionlabs.com/downloads/antlr/antlr-dotnet-tool-3.3.1.770 5.7z : In addition to the files in CSharp3Bootstrap, this includes all targets and debug symbol files (*.PDB) for all binaries. It also includes Antlr4.StringTemplate.dll and Antlr4.StringTemplate.Visualizer.dll. . Source http://www.tunnelvisionlabs.com/downloads/antlr/antlr-dotnet-source-3.3.1.7 705.7z : Includes the source tree for the Tool distribution (no binaries). You may find the following documentation for the CSharp3 target useful as well. If you installed a previous copy of the Visual Studio extensions, you'll need to uninstall them thorugh the Visual Studio 2010 Extension Manager before installing the new versions. http://www.tunnelvisionlabs.com/downloads/antlr/ANTLRCSharp3_VS2010.xps . Visual Studio project setup for ANTLR grammars with your project. . Visual Studio 2010 highlighting and IntelliSense support for ANTLR v3 grammars and highlighting support for StringTemplate v4 group files. Thank you, Sam Harwell List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31186] Re: [antlr-interest] Antlr 3.3 Generate wrong debug code C# target
Hi Max, The CSharp3 target does not currently support ANTLRWorks-style debugging. I'm planning to support debugging in a future release of the target. Thanks, Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Massimiliano Donini Sent: Monday, January 17, 2011 4:26 PM To: antlr-interest@antlr.org Subject: [antlr-interest] Antlr 3.3 Generate wrong debug code C# target Hi, i'm writing because i have some problems using Antlr 3.3, target CSharp3 with -debug option. Compile the attached grammar (taken from Antlr 3.3 examples) with -debug, ANTLR generates wrong C# code eg: * generates wrong class definition eg: public partial class Myclass : DebugAntlr.Runtime.Parser instead of Antlr.Runtime.Debug.DebugParser * doesn't genrate static member decisionCanBacktrack * set the DebugListener readonly property in the constructor Best Regards Max List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31129] Re: [antlr-interest] Antrr 3.3 template Output
Hi Max, I have never used output=template for the CSharp3 target. I'll try to set up some tests to get this working for a later release. Thanks, Sam -Original Message- From: Massimiliano Donini [mailto:max...@hotmail.it] Sent: Friday, January 07, 2011 1:53 PM To: 'ANTLR Interest' Cc: Sam Harwell Subject: Antrr 3.3 template Output Hi, with Antlr 3.3 CSharp i try to use output = template, but generated class not compile because of a creation of STAttrMap object. Looking in the ST.stg at line 132 / 139 / 146 i found 3 template wich create a new istance of STAttrMap object, but i can't found in Antlr 3.3 runtime dll. Am i missing something or is an issue?? Best regards Max List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31083] Re: [antlr-interest] the nihilistic circle hoist
Here's my analysis of the problem. First and foremost, I'm treating the rule
args as though it were written args : ID+;
Problem 1: The rule args is not always executed behind the same predicate.
This is easily corrected by adding a second rule args2 and modifying the
element rule as follows:
element : {true}? = args2;
args2 : ID+;
Problem 2: If we take a detailed look at the decision making for optional,
we see a major problem. Removing predicates and inlining, we see the
following syntactic structure:
optional : ':' ID+ ID+;
After matching one ID element for the first instance of ID+, if the next
input element is ID, the grammar must decide whether to match that with
the first instance of ID+ or to move on to the second. Without predicates,
this is a grammatical error. However, the predicates you provided clearly
indicate that on the condition true, the loop should be terminated and
move on to the next instance of ID+ (contained in the second element).
This condition *must* be checked as part of the loop condition in the rule
args.
Solution: Ignoring the only truly acceptable solution of not using syntactic
predicates, you should add the following to my list of rules about semantic
predicates.
1. A semantic predicate which unconditionally evaluates to true during
*any single execution of the grammar* behaves almost as though it weren't
included in the grammar. By removing the predicate from the rule element,
the grammar compiler reports a warning for the rule args which clearly
indicates the rule args will not behave as you expect. This is the
specific rule where the decision error was observed - the semantic predicate
suppressed a critical warning.
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ron Burk
Sent: Wednesday, January 05, 2011 7:43 AM
To: ANTLR Interest
Subject: Re: [antlr-interest] the nihilistic circle hoist
That's not a bug but a limitation of Java and most other targets; I think
the book has a good description.
If we're talking about bug#2 (skip ahead if not! :-), then...
After (re)reading the book, the only thing I could figure out you might be
referring to is the final paragraphs of Chapter 13. But those are about the
issues pretty well covered by Sam's prior post of limitations.
The predicate true does not reference anything not visible to all rules
or violate any other constraint I've heard of :-).
Maybe a more extreme example can make the problem clear:
--
// this grammar is for this simple language:
// '(' ID+ ')' ( ':' ID+ )?
grammar hoist1;
WS : (' '|'\t'|'\n'|'\r')+ {skip();};
ID : ('a'..'z')+;
start : mainprog optional? EOF;
mainprog : '(' args ')';
args : ID ID* ;
// I've decided to disable this optional construct
optional: {false}?= ':' element element; element : {true}?= args ;
--
* predicates completely safe (only true/false)
* compiles (target language has no complaints)
* works sometimes! (accepts string ( x ))
* fails other times.
(the {false} predicate is there just to make the resulting output as
disturbingly incorrect as possible; you can remove it and still get the same
wrong output).
When given an input string of ( x x ) the generated parser says:
line 1:4 extraneous input 'x' expecting ')'
The reason is that the predicate is being executed far outside its correct
context. Predicate hoisting is significantly harder than the current ANTLR
algorithm thinks it is. :-) I really don't see any way to implement it
correctly without either nontrivial grammar transformations or a runtime
stack of active predicates.
But all that's about bug#2.
Back to bug#1, where hopefully it should be much simpler for me to see if
I'm confused.
I get these results with 3.3. Hmm... it looks like the decision for FOO
FOO* and FOO+ also gets the same thing. which version are you using?
The header line of my ANTLR output says:
// $ANTLR 3.3 Nov 30, 2010 12:50:56 Simple.g 2011-01-04 12:20:14
Maybe you're saying the DFA is the same so no problem, while I'm saying
the two parsers accept different languages, so problem?
Here's the exact file I just tested:
grammar hoist1;
WS : (' '|'\t'|'\n'|'\r')+ {skip();};
ID : ('a'..'z')+;
start : element* EOF ;
element : {true}?= pre ;
pre : ID+ ;
When I give this grammar an input string of x, I get:
line 1:0 required (...)+ loop did not match anything at input 'x'
When I change change the ID+ to ID ID*, the resulting parser happily
accepts any number of space-separated x's.
Is that not the same result you get?
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this
[il-antlr-interest: 31050] Re: [antlr-interest] the nihilistic circle hoist
When a semantic predicate is involved with decision making in a grammar, the
following rules should be considered absolute:
1. The predicate cannot access any argument to any rule.
2. The predicate cannot access any variables declared within an @init{}
block of any rule.
3. The predicate cannot access any member of any attribute scope in the
grammar.
In addition, semantic predicates should be pure in the sense that they do
not introduce changes to the program state under any circumstance.
Semantic predicates may safely do the following:
1. Access immutable variables declared within the @members{} section of the
grammar. If the rule foo is the entry point to your grammar, then any
variable x which is used by a semantic predicate should be initialized
before foo() is entered, and should remain unchanged until foo() completes.
2. Call input.la(n) and input.lt(n).
Finally, like the backtrack grammar option, you should do whatever is
necessary to remove semantic predicates from a production grammar as they
introduce a significant performance penalty. As an example, consider the
following example from a grammar I wrote. My language allows function
declarations of the form function modifier* returnType? functionName,
where modifiers are effectively user-configured keywords in this context.
Originally, I had the following:
functionDefinition : 'function' ({IsFunctionModifier(input.LT(1))}? =
mods+=IDENTIFIER)* retType=IDENTIFIER? functionBody
- ^('function' ^(MODS $mods*) retType? functionBody);
I now use the following:
functionDefinition : 'function' mods+=IDENTIFIER* functionBody
- ^('function' ^(MODS {ExtractFunctionModifiers($mods)})
{ExtractFunctionReturnType($mods)} functionBody);
The methods ExtractFunctionModifiers and ExtractFunctionReturnType each
return a CommonTree. ExtractFunctionModifiers returns a tree with a nil()
root and any number of children which are the actual modifiers. If
ExtractFunctionReturnType returns null, the node is not included in the
rewrite tree, so this works for cases where the return type is omitted.
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ron Burk
Sent: Friday, December 31, 2010 11:14 PM
To: antlr-interest@antlr.org
Subject: Re: [antlr-interest] the nihilistic circle hoist
More fun with predicate hoisting:
grammar Simple;
FOO : 'foo' ;
section : element* unrelated EOF ;
element : {P1}?= pre;
pre : FOO post ;
post : FOO*;
unrelated : ':' post;
In this grammar, the generated code invokes P1 (completely outside its
intended syntactic context) while trying to recognize an unrelated
(because P1 was hoisted downward into post, and post is reachable via
a nonterminal unrelated to element).
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
