Hi Matthias, Thank you for the clarification with regards to the parser grammar and built-in functions. To me, as a relative newbie, in one way, the built-in functions seem similar to java.lang.* classes in Java, but in another way, they seem almost even more intrinsic than that.
I guess what I was mainly trying to get at is that Dml.g4 is fantastic in terms of flexibility, especially when considered in conjunction with the supporting referenced Java classes, but Nakul's grammar for IDE support is able to convey the essence of the language capabilities (such as built-in functions) in one place. Also, consider the following. The icdf() function is not in Dml.g4 since it is a built-in function. It is handled in the parser.ParameterizedBuiltinFunctionExpression class. On the other hand, as.scalar() is in Dml.g4 and in parser.BuiltinFunctionExpression. However, castAsScalar() is not in Dml.g4 but is handled similarly in parser.BuiltinFunctionExpression. As a result of this, I am now confused. If I am trying to understand what I can do with the language, do I look in Dml.g4 (such as for as.scalar()), do I look in BuiltinFunctionExpression (such as for as.scalar() and castAsScalar()), or do I look in ParameterizedBuiltinFunctionExpression (such as for icdf())? Perhaps as.scalar() needs to be removed from the grammar and as.scalar() and castAsScalar() need to be condensed to one function and I will be less confused. :-) Maybe I am overthinking things. The DML Language Reference does contain a tremendous amount of useful information with regards to the language capabilities. Deron On Sat, Jan 2, 2016 at 12:25 PM, Matthias Boehm <mbo...@us.ibm.com> wrote: > just to clarify, our parser grammar does not include builtin functions > because they are -- in contrast to keywords and language constructs -- not > part of the DML/PyDML syntax. This is important for both maintainability > and flexibility. For example, it allows you to define a variable or > user-defined function with the name 't' although there exists a builtin > function t() for transpose operations. > > Regards, > Matthias > > > [image: Inactive hide details for Deron Eriksson ---01/01/2016 11:38:27 > AM---Hi Nakul, This is very cool! I see great value in IDE tool]Deron > Eriksson ---01/01/2016 11:38:27 AM---Hi Nakul, This is very cool! I see > great value in IDE tools such as an Eclipse editor > > From: Deron Eriksson <deroneriks...@gmail.com> > To: dev@systemml.incubator.apache.org > Date: 01/01/2016 11:38 AM > Subject: Re: POC Eclipse IDE for DML > ------------------------------ > > > > Hi Nakul, > > This is very cool! I see great value in IDE tools such as an Eclipse editor > for DML/PyDML. Highlighting syntax errors helps people catch syntax > mistakes before running the script, and IDE code completion is a great > productivity boost. > > Also, I like how clean and readable the grammar is. I don't know how others > feel, but it is so clean that, if flushed out, it might be a useful > addition to the DML Language Reference. It does things such as enumerate > the built-in functions, which is handy when trying to wrap your mind around > the language. The Dml.g4 file is very powerful and useful but I have a > harder time understanding the language syntax when looking at it since > things like built-in functions aren't included in it. I wonder if there is > a way to integrate some of your work into Dml.g4? > > I haven't done Xtext before. Do you have any advice to get started if I'd > like to try this out in my Eclipse (for instance, to look at the semicolon > issue)? > > Deron > > > > On Mon, Dec 21, 2015 at 2:44 PM, Nakul Jindal <naku...@gmail.com> wrote: > > > Hi, > > > > I've been trying to build a Proof of Concept IDE for DML using Xtext. > > > > https://github.com/nakul02/sysml.dml > > > > Grammar File : > > > > > https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext > > > > Here are some screenshots: > > http://imgur.com/a/ZJyg7 > > > > The files shown are from the algorithms folder. > > > https://github.com/apache/incubator-systemml/tree/master/scripts/algorithms > > > > (There was no particular reason to have chosen these files over others) > > > > For this POC, my only goal was to get syntax highlighting to work in > > Eclipse. > > SystemML DML grammar is written in ANTLRv4 whereas Xtext uses ANTLRv3. > Some > > amount of work was needed on the grammar before it could be used it for > > Xtext. > > > > From the screenshots, you can see that eclipse thinks there are syntactic > > errors in the files. This has to do with statements ending in an optional > > semicolon. > > > > I only supplied xtext a grammar file from which to generate the IDE. I > have > > not yet worked on the generated code to add any other features. > > I could not come up with a LL(*) grammar that would let me express > optional > > semicolons at the end of statements. For the time being, the grammar > > requires semicolons at the end of statements (as you can see in the > grammar > > file : > > > > > https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext > > ) > > > > I discussed this problem with Fred(@frreiss) and he suggested 3 ways of > > dealing with this (assuming no way of modifying the grammar to fix this): > > > > 1. Support a subset of DML in the XText based tools > > 2. Modify the xtext generated tools to fix this > > 3. Write an eclipse plugin from scratch with complete control over lexer > > and parser (among other components) > > > > Option 1 requires the least amount of work. > > I am not sure between options 2 & 3, which would be more work. > > > > > > Is this useful to anyone? > > If so, are there any thoughts or suggestions on how to approach the > > optional semicolon problem? > > > > > > Nakul Jindal > > https://github.com/nakul02/ > > > > >
