BNF grammar for D?
Hi, cedet is a collection of emacs dev tools. I understand that it should be relatively easy to enhance the emacs support of D if one just provides cedet with a BNF grammar in a Bison like format, I cite: " You should choose to use the Semantic lexer/grammer format for your language if it has a deterministic grammar. Often times you can download a pre- existing BNF grammar for a language. These BNF grammars can be converted to Wisent grammar format fairly easily. " Is there anything that could be easily translated? Regards, Thomas Koch
BNF grammar for D?
There have been similar questions asked on this forum before (most recent one in June with no result: http://forum.dlang.org/thread/cafmgiz8fyv2a+scqpqteyesfsybznfy--nsxl5rqnowwna4...@mail.gmail.com?page=1), but I was curious if anyone happens to have a BNF grammar for D laying around. I've searched all over and can't seem to find anything. I'm trying to make a plugin for IntelliJ IDEA to support D (https://github.com/landaire/intelliD is what I have now utilizing DDT's lexer) but a lot of what's required to add advanced features like code folding, code completion, etc. is generally autogenerated by JetBrains's Grammar Kit plugin which only generates that stuff from a BNF grammar. I know nothing about writing language grammars but based off what I've seen it doesn't look like it'd be *too* difficult to pick up, so if anyone also has suggestions for good readings I'd appreciate that as well.
Re: BNF grammar for D?
Hi, I can give you three options to choose from. There's the official grammar (don't know what the form is called): http://dlang.org/declaration.html A Parser Expression Grammar (PEG): https://github.com/PhilippeSigaud/Pegged/blob/master/pegged/examples/dparser.d The source code comments in my project using a pseudo-BNF format. They shouldn't be too hard to decipher, but keep in mind that they're not 100% complete and exact: http://dl.dropbox.com/u/17101773/next/2/dil/doc/dil.parser.Parser.html#Parser.parseModuleDecl -- My D Compiler: http://code.google.com/p/dil
Re: BNF grammar for D?
On Fri, 2012-10-12 at 15:11 +0200, Aziz K. wrote: > Hi, > > I can give you three options to choose from. > > There's the official grammar (don't know what the form is called): > > http://dlang.org/declaration.html > > A Parser Expression Grammar (PEG): > > https://github.com/PhilippeSigaud/Pegged/blob/master/pegged/examples/dparser.d > > The source code comments in my project using a pseudo-BNF format. They > shouldn't be too hard to decipher, but keep in mind that they're not 100% > complete and exact: > > http://dl.dropbox.com/u/17101773/next/2/dil/doc/dil.parser.Parser.html#Parser.parseModuleDecl I think the official grammar rules in, let's call it K&R grammar, is the only choice here. Rather than scrape from the website is there a file in the Git repository with this grammar rule set? If so then a small program in say, Python ;-), would give a EBNF grammar representation fairly straightforwardly – read grammar ruleset, construct tree representing the ruleset, run visitor over tree to generate EBNF. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: BNF grammar for D?
On Friday, 12 October 2012 at 14:41:12 UTC, Russel Winder wrote: On Fri, 2012-10-12 at 15:11 +0200, Aziz K. wrote: Hi, I can give you three options to choose from. There's the official grammar (don't know what the form is called): http://dlang.org/declaration.html A Parser Expression Grammar (PEG): https://github.com/PhilippeSigaud/Pegged/blob/master/pegged/examples/dparser.d The source code comments in my project using a pseudo-BNF format. They shouldn't be too hard to decipher, but keep in mind that they're not 100% complete and exact: http://dl.dropbox.com/u/17101773/next/2/dil/doc/dil.parser.Parser.html#Parser.parseModuleDecl I think the official grammar rules in, let's call it K&R grammar, is the only choice here. Rather than scrape from the website is there a file in the Git repository with this grammar rule set? If so then a small program in say, Python ;-), would give a EBNF grammar representation fairly straightforwardly – read grammar ruleset, construct tree representing the ruleset, run visitor over tree to generate EBNF. My grammar knowledge is a bit rusty, but isn't EBNF only possible for LR(K) languages? Is is possible for D? -- Paulo
Re: BNF grammar for D?
On Fri, 2012-10-12 at 16:59 +0200, Paulo Pinto wrote: […] > > My grammar knowledge is a bit rusty, but isn't EBNF only possible > for LR(K) languages? Mentioning EBNF turns out to be a red herring. Indeed mention of BNF is a red herring also. What is actually needed is a Wisent grammar file. Wisent is Bison. Bison is YACC. So we are looking for an LALR(1) grammar. > Is is possible for D? Pass. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: BNF grammar for D?
On Fri, 12 Oct 2012 16:16:10 +0100 Russel Winder wrote: > On Fri, 2012-10-12 at 16:59 +0200, Paulo Pinto wrote: > […] > > > > My grammar knowledge is a bit rusty, but isn't EBNF only possible > > for LR(K) languages? > > Mentioning EBNF turns out to be a red herring. Indeed mention of BNF > is a red herring also. What is actually needed is a Wisent grammar > file. Wisent is Bison. Bison is YACC. So we are looking for an LALR(1) > grammar. > > > Is is possible for D? > > Pass. > I'm pretty sure a correct D grammar is impossible in LALR(1), unless maybe there's a way to manually resolve shift-reduce/reduce-reduce conflicts. You might be able to get close, though. LALR(k) or GLR might be possible.
Re: BNF grammar for D?
http://www.dsource.org/projects/visuald/wiki/GrammarComparison
Re: BNF grammar for D?
Nick Sabalausky wrote: > I'm pretty sure a correct D grammar is impossible in LALR(1) "a^n b^n c^n" is not LALR(1). But "a^n b^n c^m" is---if n and m are not dependent. I.e.: without specifying the restrictions for the allowed semantic checks there is no sureness for impossibilities. -manfred
Re: BNF grammar for D?
On 10/13/2012 8:02 AM, Mehrdad wrote: http://www.dsource.org/projects/visuald/wiki/GrammarComparison I haven't tried it for some time, but you can generate the text files from the current documentation with the script in http://www.dsource.org/projects/visuald/browser/grammar
Re: BNF grammar for D?
On Monday, 27 October 2014 at 22:06:28 UTC, landaire wrote: I've searched all over and can't seem to find anything. There are links to my projects in that thread. https://github.com/Hackerpilot/libdparse should help. The doc comments for the parser tell you the grammar that they implement. (The DGrammar project is based on these ddoc comments. This reminds me that I need to update it.) The official language spec on http://dlang.org/grammar.html has been getting better recently. You may want to look at that as well. I'm trying to make a plugin for IntelliJ IDEA to support D (https://github.com/landaire/intelliD is what I have now utilizing DDT's lexer) but a lot of what's required to add advanced features like code folding, code completion, etc. is generally autogenerated by JetBrains's Grammar Kit plugin which only generates that stuff from a BNF grammar. Beware that D's grammar is ambiguous in several places.
Re: BNF grammar for D?
This looks eerily familiar... I took a stab at an Intellij plugin a long time ago, and happened to pick the same name: https://github.com/elendel-/intelliD Been meaning to pick back up again, but haven't had the chance... was waiting for the Eclipse plugin code to mature, then got distracted. Feel free to take anything that looks useful. Brian Schott (Hackerpilot) has done a lot of useful work grammar wise. See: https://github.com/Hackerpilot/DGrammar An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. On Mon, Oct 27, 2014 at 3:06 PM, landaire via Digitalmars-d < digitalmars-d@puremagic.com> wrote: > There have been similar questions asked on this forum before (most recent > one in June with no result: http://forum.dlang.org/thread/CAFMGiz8Fyv2A+ > scqpqteyesfsybznfy--nsxl5rqnowwna4...@mail.gmail.com?page=1), but I was > curious if anyone happens to have a BNF grammar for D laying around. I've > searched all over and can't seem to find anything. > > I'm trying to make a plugin for IntelliJ IDEA to support D ( > https://github.com/landaire/intelliD is what I have now utilizing DDT's > lexer) but a lot of what's required to add advanced features like code > folding, code completion, etc. is generally autogenerated by JetBrains's > Grammar Kit plugin which only generates that stuff from a BNF grammar. > > I know nothing about writing language grammars but based off what I've > seen it doesn't look like it'd be *too* difficult to pick up, so if anyone > also has suggestions for good readings I'd appreciate that as well. >
Re: BNF grammar for D?
On Monday, 27 October 2014 at 22:30:15 UTC, Brian Schott wrote: On Monday, 27 October 2014 at 22:06:28 UTC, landaire wrote: I've searched all over and can't seem to find anything. There are links to my projects in that thread. Sorry, what I meant was I couldn't find a BNF grammar :) I must have overlooked libdparse but I did find https://github.com/Hackerpilot/DGrammar which helped quite a bit! But cool, thanks for mentioning the grammar comments. I was more or less hoping I wouldn't have to redo a bunch of the work that someone's already done for me by writing the grammar. Even if I do have to do it from scratch it should be a good learning experience. Beware that D's grammar is ambiguous in several places. Do you have any specific examples off the top of your head?
Re: BNF grammar for D?
> > There are links to my projects in that thread. https://github.com/ > Hackerpilot/libdparse should help. The doc comments for the parser tell > you the grammar that they implement. (The DGrammar project is based on > these ddoc comments. This reminds me that I need to update it.) The > official language spec on http://dlang.org/grammar.html has been getting > better recently. You may want to look at that as well. > > Not only was I ninja'd by the actual expert, just realized your project is explicitly based on mine... go look at Brian's stuff, it's much better than mine. >From my mucking about before, I think you'll get more mileage from using libdparse directly than trying to (re)implement a parser for the plugin. For one, you save yourself all the pain of fixing weird edge cases and keeping up to date.
Re: BNF grammar for D?
On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via Digitalmars-d wrote: This looks eerily familiar... I took a stab at an Intellij plugin a long time ago, and happened to pick the same name: https://github.com/elendel-/intelliD Been meaning to pick back up again, but haven't had the chance... was waiting for the Eclipse plugin code to mature, then got distracted. Feel free to take anything that looks useful. Brian Schott (Hackerpilot) has done a lot of useful work grammar wise. See: https://github.com/Hackerpilot/DGrammar An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. I actually forked yours originally to get it running in IDEA 14, then decided to mess with the lexer since it was causing some problems and went off from there. I saw it was somewhat inactive and tried contacting you, but couldn't find any of your contact info anywhere! Glad to see you turned up here.
Re: BNF grammar for D?
On Monday, 27 October 2014 at 22:43:55 UTC, Jeremy Powers via Digitalmars-d wrote: From my mucking about before, I think you'll get more mileage from using libdparse directly than trying to (re)implement a parser for the plugin. For one, you save yourself all the pain of fixing weird edge cases and keeping up to date. I agree, but integrating something external with IntelliJ's platform seems like kind of a PITA. It looks like a lot of stuff makes heavy use of PSI trees (see: https://confluence.jetbrains.com/display/IDEADEV/Developing+Custom+Language+Plugins+for+IntelliJ+IDEA#DevelopingCustomLanguagePluginsforIntelliJIDEA-ImplementingaParserandPSI) I'm no expert in developing IntelliJ plugins either though...
Re: BNF grammar for D?
> Glad to see you turned up here. I'm usually lurking around... > I agree, but integrating something external with IntelliJ's platform seems > like kind of a PITA. It looks like a lot of stuff makes heavy use of PSI > trees (see: https://confluence.jetbrains.com/display/IDEADEV/ > Developing+Custom+Language+Plugins+for+IntelliJ+IDEA# > DevelopingCustomLanguagePluginsforIntelliJIDEA-ImplementingaParserandPSI) > > I'm no expert in developing IntelliJ plugins either though... > The totally-not-thought-out idea I had was to have the IntelliJ tree map to the results of libdparse. So each AST (or PSI) call is just a front for the info provided by libdparse (with caching etc). Then all the work shifts away from writing a parser to writing the hairy integration bits.
Re: BNF grammar for D?
On Monday, 27 October 2014 at 22:06:28 UTC, landaire wrote: There have been similar questions asked on this forum before (most recent one in June with no result: http://forum.dlang.org/thread/cafmgiz8fyv2a+scqpqteyesfsybznfy--nsxl5rqnowwna4...@mail.gmail.com?page=1), but I was curious if anyone happens to have a BNF grammar for D laying around. I've searched all over and can't seem to find anything. I'm trying to make a plugin for IntelliJ IDEA to support D (https://github.com/landaire/intelliD is what I have now utilizing DDT's lexer) but a lot of what's required to add advanced features like code folding, code completion, etc. is generally autogenerated by JetBrains's Grammar Kit plugin which only generates that stuff from a BNF grammar. I know nothing about writing language grammars but based off what I've seen it doesn't look like it'd be *too* difficult to pick up, so if anyone also has suggestions for good readings I'd appreciate that as well. I've also been playing with getting a Intellij IDEA plugin up but it utilises DScanner. The only issue I'm having right now is for some reason its adding two elements to the Psi's tree instead of just one. Where the second has correct data but first is the correct type. Not to mention getting dscanner to output the correct line number/index ext. Fun times.
Re: BNF grammar for D?
On Monday, 27 October 2014 at 23:17:14 UTC, Rikki Cattermole wrote: On Monday, 27 October 2014 at 22:06:28 UTC, landaire wrote: I've also been playing with getting a Intellij IDEA plugin up but it utilises DScanner. The only issue I'm having right now is for some reason its adding two elements to the Psi's tree instead of just one. Where the second has correct data but first is the correct type. Not to mention getting dscanner to output the correct line number/index ext. Fun times. Is your work published on GitHub or elsewhere? I'd like to take a look at your approach.
Re: BNF grammar for D?
On Monday, 27 October 2014 at 23:17:14 UTC, Rikki Cattermole wrote: Not to mention getting dscanner to output the correct line number/index ext. Fun times. If the lexer is giving the wrong line numbers for tokens, I want to know about it. Do you have a file that reproduces this issue?
Re: BNF grammar for D?
On Monday, 27 October 2014 at 23:50:22 UTC, Brian Schott wrote: On Monday, 27 October 2014 at 23:17:14 UTC, Rikki Cattermole wrote: Not to mention getting dscanner to output the correct line number/index ext. Fun times. If the lexer is giving the wrong line numbers for tokens, I want to know about it. Do you have a file that reproduces this issue? Not wrong numbers, just not outputting them with --ast (forked and working on it as I need it). Although if it could also lex semi colons and whitespace it would be a huge help! (not needed).
Re: BNF grammar for D?
On Monday, 27 October 2014 at 23:27:20 UTC, landaire wrote: On Monday, 27 October 2014 at 23:17:14 UTC, Rikki Cattermole wrote: On Monday, 27 October 2014 at 22:06:28 UTC, landaire wrote: I've also been playing with getting a Intellij IDEA plugin up but it utilises DScanner. The only issue I'm having right now is for some reason its adding two elements to the Psi's tree instead of just one. Where the second has correct data but first is the correct type. Not to mention getting dscanner to output the correct line number/index ext. Fun times. Is your work published on GitHub or elsewhere? I'd like to take a look at your approach. Not yet. But send me an email fi...@lastname.co.nz I'll either reply by that one or a gmail account (ssl certs). We can have a chat/send it to you about it.
Re: BNF grammar for D?
On 27/10/2014 22:53, landaire wrote: On Monday, 27 October 2014 at 22:43:55 UTC, Jeremy Powers via Digitalmars-d wrote: From my mucking about before, I think you'll get more mileage from using libdparse directly than trying to (re)implement a parser for the plugin. For one, you save yourself all the pain of fixing weird edge cases and keeping up to date. I agree, but integrating something external with IntelliJ's platform seems like kind of a PITA. It looks like a lot of stuff makes heavy use of PSI trees (see: https://confluence.jetbrains.com/display/IDEADEV/Developing+Custom+Language+Plugins+for+IntelliJ+IDEA#DevelopingCustomLanguagePluginsforIntelliJIDEA-ImplementingaParserandPSI) I'm no expert in developing IntelliJ plugins either though... Have you considered the option of creating a Psi parser than creates PsiElements from the AST parsed by DDT's DTool ? That would should be fairly straightforward. With of a fork of DDT/Dtool, you can even easily modify the DDT/Dtool parser to generate those PsiElements directly. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Re: BNF grammar for D?
On 27/10/2014 22:35, Jeremy Powers via Digitalmars-d wrote: An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. Integrating DCD might work well, because autocompletion is used sporadically (and is invoked explicitly by the user). But parsing is probably invoked quite often by IntelliJ, and as such it could be tricky to integrate with libdparse (because you'd have to invoke an external process all the time) -- Bruno Medeiros https://twitter.com/brunodomedeiros
Re: BNF grammar for D?
On 27/10/2014 22:06, landaire wrote: I'm trying to make a plugin for IntelliJ IDEA to support D (https://github.com/landaire/intelliD is what I have now utilizing DDT's lexer) but a lot of what's required to add advanced features like code folding, code completion, etc. is generally autogenerated by JetBrains's Grammar Kit plugin which only generates that stuff from a BNF grammar. Nice to hear someone re-using DDT's DTool code for another tool. (it's called DTool BTW, not plugin_tooling - that's just the directory :P ) From https://github.com/landaire/intelliD: "Backported to JDK 6 as the module made heavy use of JDK 7 and JDK 6 or lower is required for IntelliJ plugin development" WUT?? That is so backwards... I even have a hard time believing that. (I'm no IntelliJ expert so I don't know for sure what is the case). But JDK 6 is getting quite old, 7 has been out for some time now, and is already superseded by 8 (and 9 is being concocted as we speak - the pace of Java changes has picked up now that Oracle took charge) Are you really *sure* you can't write IntelliJ plugins with Java code using JDK 7 and above? IntelliJ is supposed to be Java IDE that is all fresh and up-to-date! (As opposed to Eclipse, whose code base has stagnated, I readily admit that, ;'( ) -- Bruno Medeiros https://twitter.com/brunodomedeiros
Re: BNF grammar for D?
V Fri, 31 Oct 2014 11:41:58 + Bruno Medeiros via Digitalmars-d napsáno: > On 27/10/2014 22:06, landaire wrote: > > I'm trying to make a plugin for IntelliJ IDEA to support D > > (https://github.com/landaire/intelliD is what I have now utilizing > > DDT's lexer) but a lot of what's required to add advanced features > > like code folding, code completion, etc. is generally autogenerated > > by JetBrains's Grammar Kit plugin which only generates that stuff > > from a BNF grammar. > > Nice to hear someone re-using DDT's DTool code for another tool. > (it's called DTool BTW, not plugin_tooling - that's just the > directory :P ) > > From https://github.com/landaire/intelliD: > "Backported to JDK 6 as the module made heavy use of JDK 7 and JDK 6 > or lower is required for IntelliJ plugin development" > > WUT?? That is so backwards... I even have a hard time believing that. > (I'm no IntelliJ expert so I don't know for sure what is the case). > But JDK 6 is getting quite old, 7 has been out for some time now, and > is already superseded by 8 (and 9 is being concocted as we speak - > the pace of Java changes has picked up now that Oracle took charge) > > Are you really *sure* you can't write IntelliJ plugins with Java code > using JDK 7 and above? IntelliJ is supposed to be Java IDE that is > all fresh and up-to-date! (As opposed to Eclipse, whose code base has > stagnated, I readily admit that, ;'( ) > You can write intellij idea plugins with JDK 7 or even JDK 8, but that plugins will only work with IDEA running on same version of JDK. You must use same version for IDEA SDK and for IDEA itself. Currently IDEA and all plugins are compatible with JDK 6, so this is why JDK 6 is recommended. At the moment many LTS linux distros has JDK 6 (ubuntu 12.04 LTS, RedHat/CentOS 6). But ubuntu 14.04 and readhat/centos 7 are here now, so maybe jdk7 will became recommended version soon.
Re: BNF grammar for D?
On 1/11/2014 12:35 a.m., Bruno Medeiros wrote: On 27/10/2014 22:35, Jeremy Powers via Digitalmars-d wrote: An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. Integrating DCD might work well, because autocompletion is used sporadically (and is invoked explicitly by the user). But parsing is probably invoked quite often by IntelliJ, and as such it could be tricky to integrate with libdparse (because you'd have to invoke an external process all the time) From my small test file with my attempt at this, its fine to call e.g. DScanner to grab the ast. Its actually pretty fast which I was a little shocked about.
Re: BNF grammar for D?
On 31/10/2014 12:02, Daniel Kozák via Digitalmars-d wrote: You can write intellij idea plugins with JDK 7 or even JDK 8, but that plugins will only work with IDEA running on same version of JDK. You must use same version for IDEA SDK and for IDEA itself. Currently IDEA and all plugins are compatible with JDK 6, so this is why JDK 6 is recommended. Ah, but that is totally fine then. It's actually the same scenario with Eclipse: The base Eclipse platform (and most official Eclipse plugins - that is, the plugins released by eclipse.org) are made to run in JVM 6. But if you write a plugin that requires JVM 7, then you must run Eclipse with a JVM 7, naturally. (BTW, that's the case with DDT) If you guys are writing an Intellij plugin for D, requiring the user to run it in a JVM 7 is perfectly reasonable. Even JVM 8 is trivial to install. There is not point in making your (the plugin developer's) life harder when installing a JVM 7 is trivial, in any OS. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Re: BNF grammar for D?
On 31/10/2014 12:16, Rikki Cattermole wrote: On 1/11/2014 12:35 a.m., Bruno Medeiros wrote: On 27/10/2014 22:35, Jeremy Powers via Digitalmars-d wrote: An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. Integrating DCD might work well, because autocompletion is used sporadically (and is invoked explicitly by the user). But parsing is probably invoked quite often by IntelliJ, and as such it could be tricky to integrate with libdparse (because you'd have to invoke an external process all the time) From my small test file with my attempt at this, its fine to call e.g. DScanner to grab the ast. Its actually pretty fast which I was a little shocked about. How fast is "pretty fast"? Milliseconds please. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Re: BNF grammar for D?
On 8/11/2014 4:16 a.m., Bruno Medeiros wrote: On 31/10/2014 12:16, Rikki Cattermole wrote: On 1/11/2014 12:35 a.m., Bruno Medeiros wrote: On 27/10/2014 22:35, Jeremy Powers via Digitalmars-d wrote: An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. Integrating DCD might work well, because autocompletion is used sporadically (and is invoked explicitly by the user). But parsing is probably invoked quite often by IntelliJ, and as such it could be tricky to integrate with libdparse (because you'd have to invoke an external process all the time) From my small test file with my attempt at this, its fine to call e.g. DScanner to grab the ast. Its actually pretty fast which I was a little shocked about. How fast is "pretty fast"? Milliseconds please. I didn't benchmark it. So no idea.
Re: BNF grammar for D?
On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote: On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via Digitalmars-d wrote: This looks eerily familiar... I took a stab at an Intellij plugin a long time ago, and happened to pick the same name: https://github.com/elendel-/intelliD Been meaning to pick back up again, but haven't had the chance... was waiting for the Eclipse plugin code to mature, then got distracted. Feel free to take anything that looks useful. Brian Schott (Hackerpilot) has done a lot of useful work grammar wise. See: https://github.com/Hackerpilot/DGrammar An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. I actually forked yours originally to get it running in IDEA 14, then decided to mess with the lexer since it was causing some problems and went off from there. I saw it was somewhat inactive and tried contacting you, but couldn't find any of your contact info anywhere! Glad to see you turned up here. Hi guys, I have been working on an intellij plugin which is here: https://github.com/kingsleyh/DLanguage I only started learning D a couple of weeks ago and I haven't written an intellij plugin before - and also I'm not very familiar with parsing/lexing. On my first pass - I got all the infrastructure working - e.g. run configurations, project creation, file creation etc. And I'm working on a BNF and JFlex by hand - but I realise it will take some time - so I have a branch which implements the DDT parser/lexer so I could get something working while I work on the bnf - however it breaks the contextual run configurations - which is pretty much essential for the plugin to be useful. I'd like to get the DDT parser/lexer working but I'm not really sure where to go from where I am now. My DDT branch is called with_ddt. If anyone has any pointers to what I need to do next that would be very helpful - even in terms of reading or reference material. I guess it all comes down to the DParserDefinition class - as my master branch uses a FlexAdapter which gives access to the context. But in the with_ddt branch its just using the DParser - which I think needs to have the hooks into the AST tree implemented or something like that. Anyway let me know if you have any pointers or offers of help :) --K
Re: BNF grammar for D?
On Tuesday, 16 December 2014 at 23:25:02 UTC, Kingsley wrote: On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote: On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via Digitalmars-d wrote: This looks eerily familiar... I took a stab at an Intellij plugin a long time ago, and happened to pick the same name: https://github.com/elendel-/intelliD Been meaning to pick back up again, but haven't had the chance... was waiting for the Eclipse plugin code to mature, then got distracted. Feel free to take anything that looks useful. Brian Schott (Hackerpilot) has done a lot of useful work grammar wise. See: https://github.com/Hackerpilot/DGrammar An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. I actually forked yours originally to get it running in IDEA 14, then decided to mess with the lexer since it was causing some problems and went off from there. I saw it was somewhat inactive and tried contacting you, but couldn't find any of your contact info anywhere! Glad to see you turned up here. Hi guys, I have been working on an intellij plugin which is here: https://github.com/kingsleyh/DLanguage I only started learning D a couple of weeks ago and I haven't written an intellij plugin before - and also I'm not very familiar with parsing/lexing. On my first pass - I got all the infrastructure working - e.g. run configurations, project creation, file creation etc. And I'm working on a BNF and JFlex by hand - but I realise it will take some time - so I have a branch which implements the DDT parser/lexer so I could get something working while I work on the bnf - however it breaks the contextual run configurations - which is pretty much essential for the plugin to be useful. I'd like to get the DDT parser/lexer working but I'm not really sure where to go from where I am now. My DDT branch is called with_ddt. If anyone has any pointers to what I need to do next that would be very helpful - even in terms of reading or reference material. I guess it all comes down to the DParserDefinition class - as my master branch uses a FlexAdapter which gives access to the context. But in the with_ddt branch its just using the DParser - which I think needs to have the hooks into the AST tree implemented or something like that. Anyway let me know if you have any pointers or offers of help :) --K Actually I guess what I need is a way to make the PSI structure. I'm not sure how I would do that with the DDT code I have imported.
Re: BNF grammar for D?
On 17/12/2014 1:35 p.m., Kingsley wrote: On Tuesday, 16 December 2014 at 23:25:02 UTC, Kingsley wrote: On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote: On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via Digitalmars-d wrote: This looks eerily familiar... I took a stab at an Intellij plugin a long time ago, and happened to pick the same name: https://github.com/elendel-/intelliD Been meaning to pick back up again, but haven't had the chance... was waiting for the Eclipse plugin code to mature, then got distracted. Feel free to take anything that looks useful. Brian Schott (Hackerpilot) has done a lot of useful work grammar wise. See: https://github.com/Hackerpilot/DGrammar An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. I actually forked yours originally to get it running in IDEA 14, then decided to mess with the lexer since it was causing some problems and went off from there. I saw it was somewhat inactive and tried contacting you, but couldn't find any of your contact info anywhere! Glad to see you turned up here. Hi guys, I have been working on an intellij plugin which is here: https://github.com/kingsleyh/DLanguage I only started learning D a couple of weeks ago and I haven't written an intellij plugin before - and also I'm not very familiar with parsing/lexing. On my first pass - I got all the infrastructure working - e.g. run configurations, project creation, file creation etc. And I'm working on a BNF and JFlex by hand - but I realise it will take some time - so I have a branch which implements the DDT parser/lexer so I could get something working while I work on the bnf - however it breaks the contextual run configurations - which is pretty much essential for the plugin to be useful. I'd like to get the DDT parser/lexer working but I'm not really sure where to go from where I am now. My DDT branch is called with_ddt. If anyone has any pointers to what I need to do next that would be very helpful - even in terms of reading or reference material. I guess it all comes down to the DParserDefinition class - as my master branch uses a FlexAdapter which gives access to the context. But in the with_ddt branch its just using the DParser - which I think needs to have the hooks into the AST tree implemented or something like that. Anyway let me know if you have any pointers or offers of help :) --K Actually I guess what I need is a way to make the PSI structure. I'm not sure how I would do that with the DDT code I have imported. I've sent you an email reguarding my own work on getting a plugin together using Dscanner. As well as the source code. Feel free to use it.
Re: BNF grammar for D?
On Wednesday, 17 December 2014 at 03:03:59 UTC, Rikki Cattermole wrote: On 17/12/2014 1:35 p.m., Kingsley wrote: On Tuesday, 16 December 2014 at 23:25:02 UTC, Kingsley wrote: On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote: On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via Digitalmars-d wrote: This looks eerily familiar... I took a stab at an Intellij plugin a long time ago, and happened to pick the same name: https://github.com/elendel-/intelliD Been meaning to pick back up again, but haven't had the chance... was waiting for the Eclipse plugin code to mature, then got distracted. Feel free to take anything that looks useful. Brian Schott (Hackerpilot) has done a lot of useful work grammar wise. See: https://github.com/Hackerpilot/DGrammar An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. I actually forked yours originally to get it running in IDEA 14, then decided to mess with the lexer since it was causing some problems and went off from there. I saw it was somewhat inactive and tried contacting you, but couldn't find any of your contact info anywhere! Glad to see you turned up here. Hi guys, I have been working on an intellij plugin which is here: https://github.com/kingsleyh/DLanguage I only started learning D a couple of weeks ago and I haven't written an intellij plugin before - and also I'm not very familiar with parsing/lexing. On my first pass - I got all the infrastructure working - e.g. run configurations, project creation, file creation etc. And I'm working on a BNF and JFlex by hand - but I realise it will take some time - so I have a branch which implements the DDT parser/lexer so I could get something working while I work on the bnf - however it breaks the contextual run configurations - which is pretty much essential for the plugin to be useful. I'd like to get the DDT parser/lexer working but I'm not really sure where to go from where I am now. My DDT branch is called with_ddt. If anyone has any pointers to what I need to do next that would be very helpful - even in terms of reading or reference material. I guess it all comes down to the DParserDefinition class - as my master branch uses a FlexAdapter which gives access to the context. But in the with_ddt branch its just using the DParser - which I think needs to have the hooks into the AST tree implemented or something like that. Anyway let me know if you have any pointers or offers of help :) --K Actually I guess what I need is a way to make the PSI structure. I'm not sure how I would do that with the DDT code I have imported. I've sent you an email reguarding my own work on getting a plugin together using Dscanner. As well as the source code. Feel free to use it. I've successfully integrated the Psi generation stuff with the DDT parser/lexer now with a huge thanks to Rikki. So now I think I can add all the other ps related stuff - code insight, refactoring, completion etc etc Of course its the first cut - so I'll have to do some code cleanup once I understand what I'm doing! Thanks for your help and support :)
Re: BNF grammar for D?
On 17/12/2014 09:13, Kingsley wrote: On Wednesday, 17 December 2014 at 03:03:59 UTC, Rikki Cattermole wrote: On 17/12/2014 1:35 p.m., Kingsley wrote: On Tuesday, 16 December 2014 at 23:25:02 UTC, Kingsley wrote: On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote: On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via Digitalmars-d wrote: This looks eerily familiar... I took a stab at an Intellij plugin a long time ago, and happened to pick the same name: https://github.com/elendel-/intelliD Been meaning to pick back up again, but haven't had the chance... was waiting for the Eclipse plugin code to mature, then got distracted. Feel free to take anything that looks useful. Brian Schott (Hackerpilot) has done a lot of useful work grammar wise. See: https://github.com/Hackerpilot/DGrammar An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. I actually forked yours originally to get it running in IDEA 14, then decided to mess with the lexer since it was causing some problems and went off from there. I saw it was somewhat inactive and tried contacting you, but couldn't find any of your contact info anywhere! Glad to see you turned up here. Hi guys, I have been working on an intellij plugin which is here: https://github.com/kingsleyh/DLanguage I only started learning D a couple of weeks ago and I haven't written an intellij plugin before - and also I'm not very familiar with parsing/lexing. On my first pass - I got all the infrastructure working - e.g. run configurations, project creation, file creation etc. And I'm working on a BNF and JFlex by hand - but I realise it will take some time - so I have a branch which implements the DDT parser/lexer so I could get something working while I work on the bnf - however it breaks the contextual run configurations - which is pretty much essential for the plugin to be useful. I'd like to get the DDT parser/lexer working but I'm not really sure where to go from where I am now. My DDT branch is called with_ddt. If anyone has any pointers to what I need to do next that would be very helpful - even in terms of reading or reference material. I guess it all comes down to the DParserDefinition class - as my master branch uses a FlexAdapter which gives access to the context. But in the with_ddt branch its just using the DParser - which I think needs to have the hooks into the AST tree implemented or something like that. Anyway let me know if you have any pointers or offers of help :) --K Actually I guess what I need is a way to make the PSI structure. I'm not sure how I would do that with the DDT code I have imported. I've sent you an email reguarding my own work on getting a plugin together using Dscanner. As well as the source code. Feel free to use it. I've successfully integrated the Psi generation stuff with the DDT parser/lexer now with a huge thanks to Rikki. So now I think I can add all the other ps related stuff - code insight, refactoring, completion etc etc Of course its the first cut - so I'll have to do some code cleanup once I understand what I'm doing! Thanks for your help and support :) Also, if you have questions about the DDT parser/lexer, or if there are changes that might help your project, let me know, I'll be glad to add such changes (assuming they also fit well with the way DDT uses the parser/lexer). -- Bruno Medeiros https://twitter.com/brunodomedeiros
Re: BNF grammar for D?
On Wednesday, 17 December 2014 at 12:31:32 UTC, Bruno Medeiros wrote: On 17/12/2014 09:13, Kingsley wrote: On Wednesday, 17 December 2014 at 03:03:59 UTC, Rikki Cattermole wrote: On 17/12/2014 1:35 p.m., Kingsley wrote: On Tuesday, 16 December 2014 at 23:25:02 UTC, Kingsley wrote: On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote: On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via Digitalmars-d wrote: This looks eerily familiar... I took a stab at an Intellij plugin a long time ago, and happened to pick the same name: https://github.com/elendel-/intelliD Been meaning to pick back up again, but haven't had the chance... was waiting for the Eclipse plugin code to mature, then got distracted. Feel free to take anything that looks useful. Brian Schott (Hackerpilot) has done a lot of useful work grammar wise. See: https://github.com/Hackerpilot/DGrammar An interesting path to take for an intellij plugin would be to use his DCD/libdparse for all the heavy lifting, with the plugin just farming out the work to external daemon process(es). That would avoid having yet another lexer/parser implementation to keep up to date. I actually forked yours originally to get it running in IDEA 14, then decided to mess with the lexer since it was causing some problems and went off from there. I saw it was somewhat inactive and tried contacting you, but couldn't find any of your contact info anywhere! Glad to see you turned up here. Hi guys, I have been working on an intellij plugin which is here: https://github.com/kingsleyh/DLanguage I only started learning D a couple of weeks ago and I haven't written an intellij plugin before - and also I'm not very familiar with parsing/lexing. On my first pass - I got all the infrastructure working - e.g. run configurations, project creation, file creation etc. And I'm working on a BNF and JFlex by hand - but I realise it will take some time - so I have a branch which implements the DDT parser/lexer so I could get something working while I work on the bnf - however it breaks the contextual run configurations - which is pretty much essential for the plugin to be useful. I'd like to get the DDT parser/lexer working but I'm not really sure where to go from where I am now. My DDT branch is called with_ddt. If anyone has any pointers to what I need to do next that would be very helpful - even in terms of reading or reference material. I guess it all comes down to the DParserDefinition class - as my master branch uses a FlexAdapter which gives access to the context. But in the with_ddt branch its just using the DParser - which I think needs to have the hooks into the AST tree implemented or something like that. Anyway let me know if you have any pointers or offers of help :) --K Actually I guess what I need is a way to make the PSI structure. I'm not sure how I would do that with the DDT code I have imported. I've sent you an email reguarding my own work on getting a plugin together using Dscanner. As well as the source code. Feel free to use it. I've successfully integrated the Psi generation stuff with the DDT parser/lexer now with a huge thanks to Rikki. So now I think I can add all the other ps related stuff - code insight, refactoring, completion etc etc Of course its the first cut - so I'll have to do some code cleanup once I understand what I'm doing! Thanks for your help and support :) Also, if you have questions about the DDT parser/lexer, or if there are changes that might help your project, let me know, I'll be glad to add such changes (assuming they also fit well with the way DDT uses the parser/lexer). Hi Bruno, Thanks very much. I do have a couple of questions about DDT in relation to my plugin. Firstly - I'm not too familiar with parsing/lexing but at the moment the Psi Structure I have implemented that comes from the DDT parser/lexer is not in any kind of hierarchy. All the PsiElements are available but all at the same level. Is this how the DDT parser works? Or is it down to my implementation of the Parser/Lexer that wraps it to create some hierarchy. For intellij it's going to be vastly easier to have a hierarchy with nested elements in order to get hold of a structure representing a class or a function for example - in order to do things like get the start and end lines of a class definition in order to apply code folding and to use for searching for classes and stuff. Secondly - how active it the development of DDT - does it keep up with the D2 releases. --Kingsley
Re: BNF grammar for D?
Hi Bruno, Thanks very much. I do have a couple of questions about DDT in relation to my plugin. Firstly - I'm not too familiar with parsing/lexing but at the moment the Psi Structure I have implemented that comes from the DDT parser/lexer is not in any kind of hierarchy. All the PsiElements are available but all at the same level. Is this how the DDT parser works? Or is it down to my implementation of the Parser/Lexer that wraps it to create some hierarchy. For intellij it's going to be vastly easier to have a hierarchy with nested elements in order to get hold of a structure representing a class or a function for example - in order to do things like get the start and end lines of a class definition in order to apply code folding and to use for searching for classes and stuff. Secondly - how active it the development of DDT - does it keep up with the D2 releases. --Kingsley After doing a bit more research it looks like I have to create the psi hierarchy myself - my current psi structure is flat because I'm just converting the DeeTokens into PsiElements directly. I've still got some experimentation to do. On the plus side I implemented commenting, code folding but everything else needs a psi hierarchy
Re: BNF grammar for D?
On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley wrote: Hi Bruno, Thanks very much. I do have a couple of questions about DDT in relation to my plugin. Firstly - I'm not too familiar with parsing/lexing but at the moment the Psi Structure I have implemented that comes from the DDT parser/lexer is not in any kind of hierarchy. All the PsiElements are available but all at the same level. Is this how the DDT parser works? Or is it down to my implementation of the Parser/Lexer that wraps it to create some hierarchy. For intellij it's going to be vastly easier to have a hierarchy with nested elements in order to get hold of a structure representing a class or a function for example - in order to do things like get the start and end lines of a class definition in order to apply code folding and to use for searching for classes and stuff. Secondly - how active it the development of DDT - does it keep up with the D2 releases. --Kingsley After doing a bit more research it looks like I have to create the psi hierarchy myself - my current psi structure is flat because I'm just converting the DeeTokens into PsiElements directly. I've still got some experimentation to do. On the plus side I implemented commenting, code folding but everything else needs a psi hierarchy I've done some more investigation and I do need to build the parser myself in order to create the various constructs. I've made a start but I haven't gotten very far yet because I don't fully understand the correct way to proceed. I also had a look at using the DeeParser - because it already does most of what I want. However the intellij plugin wants a PsiParser which returns an intellij ASTNode in the primary parse method. I can't see an easy way to hook this up with DeeParser because the ParsedResult although had a node method on it - gives back the wrong type of ASTNode. Any pointers on how I might get the DeeParser to interface to an intellij ASTNode would be appreciated.
Re: BNF grammar for D?
On 19/12/2014 10:19 a.m., Kingsley wrote: On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley wrote: Hi Bruno, Thanks very much. I do have a couple of questions about DDT in relation to my plugin. Firstly - I'm not too familiar with parsing/lexing but at the moment the Psi Structure I have implemented that comes from the DDT parser/lexer is not in any kind of hierarchy. All the PsiElements are available but all at the same level. Is this how the DDT parser works? Or is it down to my implementation of the Parser/Lexer that wraps it to create some hierarchy. For intellij it's going to be vastly easier to have a hierarchy with nested elements in order to get hold of a structure representing a class or a function for example - in order to do things like get the start and end lines of a class definition in order to apply code folding and to use for searching for classes and stuff. Secondly - how active it the development of DDT - does it keep up with the D2 releases. --Kingsley After doing a bit more research it looks like I have to create the psi hierarchy myself - my current psi structure is flat because I'm just converting the DeeTokens into PsiElements directly. I've still got some experimentation to do. On the plus side I implemented commenting, code folding but everything else needs a psi hierarchy I've done some more investigation and I do need to build the parser myself in order to create the various constructs. I've made a start but I haven't gotten very far yet because I don't fully understand the correct way to proceed. I also had a look at using the DeeParser - because it already does most of what I want. However the intellij plugin wants a PsiParser which returns an intellij ASTNode in the primary parse method. I can't see an easy way to hook this up with DeeParser because the ParsedResult although had a node method on it - gives back the wrong type of ASTNode. Any pointers on how I might get the DeeParser to interface to an intellij ASTNode would be appreciated. Read my codebase again, it'll answer a lot of questions. Your parser is different, but what it produces shouldn't be. and yes it supports hierarchies.
Re: BNF grammar for D?
On Friday, 19 December 2014 at 02:53:02 UTC, Rikki Cattermole wrote: On 19/12/2014 10:19 a.m., Kingsley wrote: On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley wrote: Hi Bruno, Thanks very much. I do have a couple of questions about DDT in relation to my plugin. Firstly - I'm not too familiar with parsing/lexing but at the moment the Psi Structure I have implemented that comes from the DDT parser/lexer is not in any kind of hierarchy. All the PsiElements are available but all at the same level. Is this how the DDT parser works? Or is it down to my implementation of the Parser/Lexer that wraps it to create some hierarchy. For intellij it's going to be vastly easier to have a hierarchy with nested elements in order to get hold of a structure representing a class or a function for example - in order to do things like get the start and end lines of a class definition in order to apply code folding and to use for searching for classes and stuff. Secondly - how active it the development of DDT - does it keep up with the D2 releases. --Kingsley After doing a bit more research it looks like I have to create the psi hierarchy myself - my current psi structure is flat because I'm just converting the DeeTokens into PsiElements directly. I've still got some experimentation to do. On the plus side I implemented commenting, code folding but everything else needs a psi hierarchy I've done some more investigation and I do need to build the parser myself in order to create the various constructs. I've made a start but I haven't gotten very far yet because I don't fully understand the correct way to proceed. I also had a look at using the DeeParser - because it already does most of what I want. However the intellij plugin wants a PsiParser which returns an intellij ASTNode in the primary parse method. I can't see an easy way to hook this up with DeeParser because the ParsedResult although had a node method on it - gives back the wrong type of ASTNode. Any pointers on how I might get the DeeParser to interface to an intellij ASTNode would be appreciated. Read my codebase again, it'll answer a lot of questions. Your parser is different, but what it produces shouldn't be. and yes it supports hierarchies. Hi So finally after a lot of wrestling with the internals of intellij I finally managed to get a working parser implementation that produces a psi hierarchy based on the DeeParser from the ddt code. The main issue was that Intellij only wants you to create a parser using their toolset - which is either with a BNF grammar that you can then generate the parser - or with a hand written parser. Since I'm already using the DDT lexer and there is a perfectly good DDT parser as well - I just wanted to re-use the DDT parser. However Intellij does not provide any way to create a custom AST/PSI structure or use an external parser. So I basically had to wrap the DeeParse inside the Intellij parser and sync them up programmatically. It's not the most efficient way in the world but it at least works. In the long term I will write a BNF grammar for Intellij (using their toolkit) but I can see that will take me several months so this is a quick way to get the plugin up and running with all the power of intellij extras without spending several months stuck learning all about the complexities of grammar parsing and lexing. Thanks very much for you help. Once I get a bit more of the cool stuff done I will release the plugin.
Re: BNF grammar for D?
On Sunday, 21 December 2014 at 00:34:06 UTC, Kingsley wrote: On Friday, 19 December 2014 at 02:53:02 UTC, Rikki Cattermole wrote: On 19/12/2014 10:19 a.m., Kingsley wrote: On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley wrote: Hi Bruno, Thanks very much. I do have a couple of questions about DDT in relation to my plugin. Firstly - I'm not too familiar with parsing/lexing but at the moment the Psi Structure I have implemented that comes from the DDT parser/lexer is not in any kind of hierarchy. All the PsiElements are available but all at the same level. Is this how the DDT parser works? Or is it down to my implementation of the Parser/Lexer that wraps it to create some hierarchy. For intellij it's going to be vastly easier to have a hierarchy with nested elements in order to get hold of a structure representing a class or a function for example - in order to do things like get the start and end lines of a class definition in order to apply code folding and to use for searching for classes and stuff. Secondly - how active it the development of DDT - does it keep up with the D2 releases. --Kingsley After doing a bit more research it looks like I have to create the psi hierarchy myself - my current psi structure is flat because I'm just converting the DeeTokens into PsiElements directly. I've still got some experimentation to do. On the plus side I implemented commenting, code folding but everything else needs a psi hierarchy I've done some more investigation and I do need to build the parser myself in order to create the various constructs. I've made a start but I haven't gotten very far yet because I don't fully understand the correct way to proceed. I also had a look at using the DeeParser - because it already does most of what I want. However the intellij plugin wants a PsiParser which returns an intellij ASTNode in the primary parse method. I can't see an easy way to hook this up with DeeParser because the ParsedResult although had a node method on it - gives back the wrong type of ASTNode. Any pointers on how I might get the DeeParser to interface to an intellij ASTNode would be appreciated. Read my codebase again, it'll answer a lot of questions. Your parser is different, but what it produces shouldn't be. and yes it supports hierarchies. Hi So finally after a lot of wrestling with the internals of intellij I finally managed to get a working parser implementation that produces a psi hierarchy based on the DeeParser from the ddt code. The main issue was that Intellij only wants you to create a parser using their toolset - which is either with a BNF grammar that you can then generate the parser - or with a hand written parser. Since I'm already using the DDT lexer and there is a perfectly good DDT parser as well - I just wanted to re-use the DDT parser. Hi Bruno - would be easy to return the list of tokens included for each node in the DeeParser? However Intellij does not provide any way to create a custom AST/PSI structure or use an external parser. So I basically had to wrap the DeeParse inside the Intellij parser and sync them up programmatically. It's not the most efficient way in the world but it at least works. In the long term I will write a BNF grammar for Intellij (using their toolkit) but I can see that will take me several months so this is a quick way to get the plugin up and running with all the power of intellij extras without spending several months stuck learning all about the complexities of grammar parsing and lexing. Thanks very much for you help. Once I get a bit more of the cool stuff done I will release the plugin.
Re: BNF grammar for D?
On 17/12/2014 17:19, Kingsley wrote: Secondly - how active it the development of DDT - does it keep up with the D2 releases. Yes, it keeps up, and I plan to keep that up for the foreseeable future. (Since language grammar changes are fairly uncommon, and easy to implement when they happen). -- Bruno Medeiros https://twitter.com/brunodomedeiros
Re: BNF grammar for D?
On 21/12/2014 00:34, Kingsley wrote: On Friday, 19 December 2014 at 02:53:02 UTC, Rikki Cattermole wrote: On 19/12/2014 10:19 a.m., Kingsley wrote: On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley wrote: Hi Bruno, Thanks very much. I do have a couple of questions about DDT in relation to my plugin. Firstly - I'm not too familiar with parsing/lexing but at the moment the Psi Structure I have implemented that comes from the DDT parser/lexer is not in any kind of hierarchy. All the PsiElements are available but all at the same level. Is this how the DDT parser works? Or is it down to my implementation of the Parser/Lexer that wraps it to create some hierarchy. For intellij it's going to be vastly easier to have a hierarchy with nested elements in order to get hold of a structure representing a class or a function for example - in order to do things like get the start and end lines of a class definition in order to apply code folding and to use for searching for classes and stuff. Secondly - how active it the development of DDT - does it keep up with the D2 releases. --Kingsley After doing a bit more research it looks like I have to create the psi hierarchy myself - my current psi structure is flat because I'm just converting the DeeTokens into PsiElements directly. I've still got some experimentation to do. On the plus side I implemented commenting, code folding but everything else needs a psi hierarchy I've done some more investigation and I do need to build the parser myself in order to create the various constructs. I've made a start but I haven't gotten very far yet because I don't fully understand the correct way to proceed. I also had a look at using the DeeParser - because it already does most of what I want. However the intellij plugin wants a PsiParser which returns an intellij ASTNode in the primary parse method. I can't see an easy way to hook this up with DeeParser because the ParsedResult although had a node method on it - gives back the wrong type of ASTNode. Any pointers on how I might get the DeeParser to interface to an intellij ASTNode would be appreciated. Read my codebase again, it'll answer a lot of questions. Your parser is different, but what it produces shouldn't be. and yes it supports hierarchies. Hi So finally after a lot of wrestling with the internals of intellij I finally managed to get a working parser implementation that produces a psi hierarchy based on the DeeParser from the ddt code. The main issue was that Intellij only wants you to create a parser using their toolset - which is either with a BNF grammar that you can then generate the parser - or with a hand written parser. Since I'm already using the DDT lexer and there is a perfectly good DDT parser as well - I just wanted to re-use the DDT parser. However Intellij does not provide any way to create a custom AST/PSI structure or use an external parser. So I basically had to wrap the DeeParse inside the Intellij parser and sync them up programmatically. It's not the most efficient way in the world but it at least works. In the long term I will write a BNF grammar for Intellij (using their toolkit) but I can see that will take me several months so this is a quick way to get the plugin up and running with all the power of intellij extras without spending several months stuck learning all about the complexities of grammar parsing and lexing. Thanks very much for you help. Once I get a bit more of the cool stuff done I will release the plugin. Again, I'm not familiar with Intellij internals or the PSI structure, so I don't know how the DDT parser can be adapted to that. However, I do suspect there should be a way to create a PSI structure from the DDT ASTNode structure. PS: Sorry for the long delay in replying, I don't often check the digitalmars.D newsgroup/forum, and can sometimes be behind on the posts there. If you want to grab my attention, better to post on digitalmars.D.ide as I keep a closer eye on that newsgroup. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Re: BNF grammar for D?
On 22/12/2014 11:44, Kingsley wrote: Hi Bruno - would be easy to return the list of tokens included for each node in the DeeParser? You can create an utility method that does that, if you have a DeeParseResult. The DeeParseResult has a 'tokenList' member, ordered by the source range. With a node's source range, you can do a binary search in that list to find the corresponding tokens. For more convenience, I guess DeeParser could be modified so that this information - in the form of a sublist of 'tokenList' - would be stored directly in each ASTNode, in the 'data' field. This way one would not need to provide the DeeParseResult directly. -- Bruno Medeiros https://twitter.com/brunodomedeiros
