Re: [julia-users] Re: @DSL Domain code
If you want DSLs to have arbitrary syntax, then you need to put them inside of strings or the moral equivalent thereof (e.g. Julia's backtick syntax for commands). If you're ok with the DSL using Julia syntax, then you can use a macro. Anything besides that ends up delving into making Julia's grammar extensible somehow, which just isn't going to happen – parsers aren't really composable http://tratt.net/laurie/blog/entries/parsing_the_solved_problem_that_isnt like this. On Sun, Nov 16, 2014 at 7:30 PM, Jeff Waller truth...@gmail.com wrote: Its just something inside me that rebels against having code inside strings. Yea, I don't like it either. If feels half-done. New token? @@
Re: [julia-users] Re: @DSL Domain code
No! The expression between *begin ... end* must still be valid Julia syntax, which is why you need to tell Julia, not to parse it into a Julia AST, but into some intermediate representation. @Scala begin person match { ERROR: syntax: extra token match after end of expression 2014-11-16 16:18 GMT+01:00 Johan Sigfrids johan.sigfr...@gmail.com: Would it be impossible to simply have something like: @Scala begin person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer * case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann * case* *_* = Unknown Person } end Without having to have the multiline string thing or DSL call? On Saturday, November 15, 2014 11:55:15 PM UTC+2, Simon Danisch wrote: Hi, I was thinking about Domain Specific Languages lately from different perspectives and concluded, that we could easily make Julia an awesome home for DSL's. (Domain Specific Languages) Why would Julia benefit from a good infrastructure for DSL's? *1.)* Rapid prototyping of language concepts. I was considering Mauro's trait implementation lately, and I didn't want to use it, because it just looked a little cumbersome (no offense, its just a little difficult with only macros!). If he was able to implement the prototype first with a DSL, he could have already created a syntax prototype for it and thinks could have looked more concise and inviting. *2.)* Writing in different languages inside Julia, for example Assebler, OpenCL kernels, OpenGL shader, etc... function foo(a::Float32, b::Float32) @DSL Assembler(a, b) pushRBP mov RBP, RSP vaddss XMM0, XMM0, XMM1 pop RBP ret end I know you might ask, why this is a good idea, as LLVM should be tuned, to emit the best native code. But lets assume that it doesn't! Then, a person can already prototype how the code emitted by LLVM should look like, and another person, who probably knows LLVM a lot better, can do the appropriate changes to LLVM/Julia. *3.)* Classic use cases like a specialized UI languages, first order logic, other mathematical constructs, scripting, etc... *4.)* Stealing nice syntactic concepts from other language, to see if they give Julia any advantages, without a big hassle. function isspecificperson(person) @DSL Scala person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer *case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann *case* *_* = Unknown Person } *Possible disadvantages: * If widely used, code will get harder to read, as you need to learn all the crazy DSL's users are creating. But this will happen anyways, and it's probably better to do it in an orderly fashion than ;) *Implementation Sketch:* macro DSL(name, text) tokens = *dsltokenizer*(DSLTokens{name}(), text)::DSLTokens{name} dslast = *generate_ast*(tokens)::AST{name} return *dsl*(dslast)::AST{:Julia} end # Default implementations: *dsltokenizer*(::DSLTokens, text) = ... # Default probably with Jake Bolewkis Lexer?! *generate_ast*(text::DSLTokens) = ... *dsl*(ast::AST) = ... #Depending on the complexity of your DSL, overwrite any of these stages, to implement your own DSL, otherwise use default It seems like Jake Bolewski has already implemented a lot to make this work. Probably it would be nice to integrate OpenCL kernel code like this ;) My hope would also be, to pair this with meta data on the different stages, to make it very easy to supply correct syntax highlighting/correction for the different DSL's. Creating the ast and tokenizing things otherwise needs to be done twice, one time for the system and one time for an IDE. Or are there currently simple ways in Julia, to determine where tokens are in a string, what scopes there are and what kind of attribute println is in println(1234) ? I haven't found them yet. Most of the things you would currently need to implement for this, will be redundant to parse and are then volatile to changes in the language. Well these are just some thoughts I recently had, feel free to evaluate this and/or judge if this is something we really want! I won't implement this anytime soon, but maybe someone searching for a bachelor/master thesis comes to the rescue? Who knows... Cheers, Simon
Re: [julia-users] Re: @DSL Domain code
No! The expression between *begin ... end* must still be valid Julia syntax, which is why you need to tell Julia, not to parse it into a Julia AST, but into some intermediate representation. Well, in the examples you showed the command block is passed as string anyway, so the more accurate analogy is: julia @Scala blah But there is already a nicer way to do this: julia macro Scala_mstr(args) ... end then you can do this directly and the contents of the string will be passed to the macro (as a raw string): julia Scala ... (e.g. this is used in Keno's Cxx.jl for embedding C++ code as strings) This seems like a good solution, except that some people might not like using string blocks in such a way. Personally I think it's fine: there must be a reserved block start/end pair one way or another. On Sun, Nov 16, 2014 at 10:26 AM, Simon Danisch sdani...@gmail.com wrote: No! The expression between *begin ... end* must still be valid Julia syntax, which is why you need to tell Julia, not to parse it into a Julia AST, but into some intermediate representation. @Scala begin person match { ERROR: syntax: extra token match after end of expression 2014-11-16 16:18 GMT+01:00 Johan Sigfrids johan.sigfr...@gmail.com: Would it be impossible to simply have something like: @Scala begin person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer * case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann * case* *_* = Unknown Person } end Without having to have the multiline string thing or DSL call? On Saturday, November 15, 2014 11:55:15 PM UTC+2, Simon Danisch wrote: Hi, I was thinking about Domain Specific Languages lately from different perspectives and concluded, that we could easily make Julia an awesome home for DSL's. (Domain Specific Languages) Why would Julia benefit from a good infrastructure for DSL's? *1.)* Rapid prototyping of language concepts. I was considering Mauro's trait implementation lately, and I didn't want to use it, because it just looked a little cumbersome (no offense, its just a little difficult with only macros!). If he was able to implement the prototype first with a DSL, he could have already created a syntax prototype for it and thinks could have looked more concise and inviting. *2.)* Writing in different languages inside Julia, for example Assebler, OpenCL kernels, OpenGL shader, etc... function foo(a::Float32, b::Float32) @DSL Assembler(a, b) pushRBP mov RBP, RSP vaddss XMM0, XMM0, XMM1 pop RBP ret end I know you might ask, why this is a good idea, as LLVM should be tuned, to emit the best native code. But lets assume that it doesn't! Then, a person can already prototype how the code emitted by LLVM should look like, and another person, who probably knows LLVM a lot better, can do the appropriate changes to LLVM/Julia. *3.)* Classic use cases like a specialized UI languages, first order logic, other mathematical constructs, scripting, etc... *4.)* Stealing nice syntactic concepts from other language, to see if they give Julia any advantages, without a big hassle. function isspecificperson(person) @DSL Scala person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer *case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann *case* *_* = Unknown Person } *Possible disadvantages: * If widely used, code will get harder to read, as you need to learn all the crazy DSL's users are creating. But this will happen anyways, and it's probably better to do it in an orderly fashion than ;) *Implementation Sketch:* macro DSL(name, text) tokens = *dsltokenizer*(DSLTokens{name}(), text)::DSLTokens{name} dslast = *generate_ast*(tokens)::AST{name} return *dsl*(dslast)::AST{:Julia} end # Default implementations: *dsltokenizer*(::DSLTokens, text) = ... # Default probably with Jake Bolewkis Lexer?! *generate_ast*(text::DSLTokens) = ... *dsl*(ast::AST) = ... #Depending on the complexity of your DSL, overwrite any of these stages, to implement your own DSL, otherwise use default It seems like Jake Bolewski has already implemented a lot to make this work. Probably it would be nice to integrate OpenCL kernel code like this ;) My hope would also be, to pair this with meta data on the different stages, to make it very easy to supply correct syntax highlighting/correction for the different DSL's. Creating the ast and tokenizing things otherwise needs to be done twice, one time for the system and one time for an IDE. Or are there currently simple ways in Julia, to determine where tokens are in a string, what scopes there are and what kind of attribute println is in println(1234) ? I haven't found them yet. Most of the things you would currently need to implement for this, will be redundant to parse and are then volatile to changes in
Re: [julia-users] Re: @DSL Domain code
I'd argue (and in fact we have in the academic literature) that JuMP.jl is a DSL for mathematical programming, e.g. m = Model() @defVar(m, x[1:5], Bin) @addConstraint(m, sum{x[i], i=1:5; iseven(i)} = 2) solve(m) Notice the sum{ } On Sunday, November 16, 2014 10:45:57 AM UTC-5, Isaiah wrote: No! The expression between *begin ... end* must still be valid Julia syntax, which is why you need to tell Julia, not to parse it into a Julia AST, but into some intermediate representation. Well, in the examples you showed the command block is passed as string anyway, so the more accurate analogy is: julia @Scala blah But there is already a nicer way to do this: julia macro Scala_mstr(args) ... end then you can do this directly and the contents of the string will be passed to the macro (as a raw string): julia Scala ... (e.g. this is used in Keno's Cxx.jl for embedding C++ code as strings) This seems like a good solution, except that some people might not like using string blocks in such a way. Personally I think it's fine: there must be a reserved block start/end pair one way or another. On Sun, Nov 16, 2014 at 10:26 AM, Simon Danisch sdan...@gmail.com javascript: wrote: No! The expression between *begin ... end* must still be valid Julia syntax, which is why you need to tell Julia, not to parse it into a Julia AST, but into some intermediate representation. @Scala begin person match { ERROR: syntax: extra token match after end of expression 2014-11-16 16:18 GMT+01:00 Johan Sigfrids johan.s...@gmail.com javascript:: Would it be impossible to simply have something like: @Scala begin person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer * case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann * case* *_* = Unknown Person } end Without having to have the multiline string thing or DSL call? On Saturday, November 15, 2014 11:55:15 PM UTC+2, Simon Danisch wrote: Hi, I was thinking about Domain Specific Languages lately from different perspectives and concluded, that we could easily make Julia an awesome home for DSL's. (Domain Specific Languages) Why would Julia benefit from a good infrastructure for DSL's? *1.)* Rapid prototyping of language concepts. I was considering Mauro's trait implementation lately, and I didn't want to use it, because it just looked a little cumbersome (no offense, its just a little difficult with only macros!). If he was able to implement the prototype first with a DSL, he could have already created a syntax prototype for it and thinks could have looked more concise and inviting. *2.)* Writing in different languages inside Julia, for example Assebler, OpenCL kernels, OpenGL shader, etc... function foo(a::Float32, b::Float32) @DSL Assembler(a, b) pushRBP mov RBP, RSP vaddss XMM0, XMM0, XMM1 pop RBP ret end I know you might ask, why this is a good idea, as LLVM should be tuned, to emit the best native code. But lets assume that it doesn't! Then, a person can already prototype how the code emitted by LLVM should look like, and another person, who probably knows LLVM a lot better, can do the appropriate changes to LLVM/Julia. *3.)* Classic use cases like a specialized UI languages, first order logic, other mathematical constructs, scripting, etc... *4.)* Stealing nice syntactic concepts from other language, to see if they give Julia any advantages, without a big hassle. function isspecificperson(person) @DSL Scala person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer *case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann *case* *_* = Unknown Person } *Possible disadvantages: * If widely used, code will get harder to read, as you need to learn all the crazy DSL's users are creating. But this will happen anyways, and it's probably better to do it in an orderly fashion than ;) *Implementation Sketch:* macro DSL(name, text) tokens = *dsltokenizer*(DSLTokens{name}(), text)::DSLTokens{name} dslast = *generate_ast*(tokens)::AST{name} return *dsl*(dslast)::AST{:Julia} end # Default implementations: *dsltokenizer*(::DSLTokens, text) = ... # Default probably with Jake Bolewkis Lexer?! *generate_ast*(text::DSLTokens) = ... *dsl*(ast::AST) = ... #Depending on the complexity of your DSL, overwrite any of these stages, to implement your own DSL, otherwise use default It seems like Jake Bolewski has already implemented a lot to make this work. Probably it would be nice to integrate OpenCL kernel code like this ;) My hope would also be, to pair this with meta data on the different stages, to make it very easy to supply correct syntax highlighting/correction for the different DSL's. Creating the ast and tokenizing things otherwise needs to be done twice, one
Re: [julia-users] Re: @DSL Domain code
Your right. That is a problem. Is there no way to extend Julia so that you can create a macro that automatically uses a custom parser? On Sunday, November 16, 2014 5:26:26 PM UTC+2, Simon Danisch wrote: No! The expression between *begin ... end* must still be valid Julia syntax, which is why you need to tell Julia, not to parse it into a Julia AST, but into some intermediate representation. @Scala begin person match { ERROR: syntax: extra token match after end of expression 2014-11-16 16:18 GMT+01:00 Johan Sigfrids johan.s...@gmail.com javascript:: Would it be impossible to simply have something like: @Scala begin person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer * case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann * case* *_* = Unknown Person } end Without having to have the multiline string thing or DSL call? On Saturday, November 15, 2014 11:55:15 PM UTC+2, Simon Danisch wrote: Hi, I was thinking about Domain Specific Languages lately from different perspectives and concluded, that we could easily make Julia an awesome home for DSL's. (Domain Specific Languages) Why would Julia benefit from a good infrastructure for DSL's? *1.)* Rapid prototyping of language concepts. I was considering Mauro's trait implementation lately, and I didn't want to use it, because it just looked a little cumbersome (no offense, its just a little difficult with only macros!). If he was able to implement the prototype first with a DSL, he could have already created a syntax prototype for it and thinks could have looked more concise and inviting. *2.)* Writing in different languages inside Julia, for example Assebler, OpenCL kernels, OpenGL shader, etc... function foo(a::Float32, b::Float32) @DSL Assembler(a, b) pushRBP mov RBP, RSP vaddss XMM0, XMM0, XMM1 pop RBP ret end I know you might ask, why this is a good idea, as LLVM should be tuned, to emit the best native code. But lets assume that it doesn't! Then, a person can already prototype how the code emitted by LLVM should look like, and another person, who probably knows LLVM a lot better, can do the appropriate changes to LLVM/Julia. *3.)* Classic use cases like a specialized UI languages, first order logic, other mathematical constructs, scripting, etc... *4.)* Stealing nice syntactic concepts from other language, to see if they give Julia any advantages, without a big hassle. function isspecificperson(person) @DSL Scala person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer *case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann *case* *_* = Unknown Person } *Possible disadvantages: * If widely used, code will get harder to read, as you need to learn all the crazy DSL's users are creating. But this will happen anyways, and it's probably better to do it in an orderly fashion than ;) *Implementation Sketch:* macro DSL(name, text) tokens = *dsltokenizer*(DSLTokens{name}(), text)::DSLTokens{name} dslast = *generate_ast*(tokens)::AST{name} return *dsl*(dslast)::AST{:Julia} end # Default implementations: *dsltokenizer*(::DSLTokens, text) = ... # Default probably with Jake Bolewkis Lexer?! *generate_ast*(text::DSLTokens) = ... *dsl*(ast::AST) = ... #Depending on the complexity of your DSL, overwrite any of these stages, to implement your own DSL, otherwise use default It seems like Jake Bolewski has already implemented a lot to make this work. Probably it would be nice to integrate OpenCL kernel code like this ;) My hope would also be, to pair this with meta data on the different stages, to make it very easy to supply correct syntax highlighting/correction for the different DSL's. Creating the ast and tokenizing things otherwise needs to be done twice, one time for the system and one time for an IDE. Or are there currently simple ways in Julia, to determine where tokens are in a string, what scopes there are and what kind of attribute println is in println(1234) ? I haven't found them yet. Most of the things you would currently need to implement for this, will be redundant to parse and are then volatile to changes in the language. Well these are just some thoughts I recently had, feel free to evaluate this and/or judge if this is something we really want! I won't implement this anytime soon, but maybe someone searching for a bachelor/master thesis comes to the rescue? Who knows... Cheers, Simon
Re: [julia-users] Re: @DSL Domain code
Well, the DSL macro I sketched out would allow for that. Maybe, if time comes and people think this is a good idea, this could become an extension to normal macros? But first of all, a prototype is needed anyways... How deeply the prototype will be integrated later on depends on the feedback, I guess. 2014-11-16 20:01 GMT+01:00 Johan Sigfrids johan.sigfr...@gmail.com: Your right. That is a problem. Is there no way to extend Julia so that you can create a macro that automatically uses a custom parser? On Sunday, November 16, 2014 5:26:26 PM UTC+2, Simon Danisch wrote: No! The expression between *begin ... end* must still be valid Julia syntax, which is why you need to tell Julia, not to parse it into a Julia AST, but into some intermediate representation. @Scala begin person match { ERROR: syntax: extra token match after end of expression 2014-11-16 16:18 GMT+01:00 Johan Sigfrids johan.s...@gmail.com: Would it be impossible to simply have something like: @Scala begin person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer * case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann * case* *_* = Unknown Person } end Without having to have the multiline string thing or DSL call? On Saturday, November 15, 2014 11:55:15 PM UTC+2, Simon Danisch wrote: Hi, I was thinking about Domain Specific Languages lately from different perspectives and concluded, that we could easily make Julia an awesome home for DSL's. (Domain Specific Languages) Why would Julia benefit from a good infrastructure for DSL's? *1.)* Rapid prototyping of language concepts. I was considering Mauro's trait implementation lately, and I didn't want to use it, because it just looked a little cumbersome (no offense, its just a little difficult with only macros!). If he was able to implement the prototype first with a DSL, he could have already created a syntax prototype for it and thinks could have looked more concise and inviting. *2.)* Writing in different languages inside Julia, for example Assebler, OpenCL kernels, OpenGL shader, etc... function foo(a::Float32, b::Float32) @DSL Assembler(a, b) pushRBP mov RBP, RSP vaddss XMM0, XMM0, XMM1 pop RBP ret end I know you might ask, why this is a good idea, as LLVM should be tuned, to emit the best native code. But lets assume that it doesn't! Then, a person can already prototype how the code emitted by LLVM should look like, and another person, who probably knows LLVM a lot better, can do the appropriate changes to LLVM/Julia. *3.)* Classic use cases like a specialized UI languages, first order logic, other mathematical constructs, scripting, etc... *4.)* Stealing nice syntactic concepts from other language, to see if they give Julia any advantages, without a big hassle. function isspecificperson(person) @DSL Scala person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer *case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann *case* *_* = Unknown Person } *Possible disadvantages: * If widely used, code will get harder to read, as you need to learn all the crazy DSL's users are creating. But this will happen anyways, and it's probably better to do it in an orderly fashion than ;) *Implementation Sketch:* macro DSL(name, text) tokens = *dsltokenizer*(DSLTokens{name}(), text)::DSLTokens{name} dslast = *generate_ast*(tokens)::AST{name} return *dsl*(dslast)::AST{:Julia} end # Default implementations: *dsltokenizer*(::DSLTokens, text) = ... # Default probably with Jake Bolewkis Lexer?! *generate_ast*(text::DSLTokens) = ... *dsl*(ast::AST) = ... #Depending on the complexity of your DSL, overwrite any of these stages, to implement your own DSL, otherwise use default It seems like Jake Bolewski has already implemented a lot to make this work. Probably it would be nice to integrate OpenCL kernel code like this ;) My hope would also be, to pair this with meta data on the different stages, to make it very easy to supply correct syntax highlighting/correction for the different DSL's. Creating the ast and tokenizing things otherwise needs to be done twice, one time for the system and one time for an IDE. Or are there currently simple ways in Julia, to determine where tokens are in a string, what scopes there are and what kind of attribute println is in println(1234) ? I haven't found them yet. Most of the things you would currently need to implement for this, will be redundant to parse and are then volatile to changes in the language. Well these are just some thoughts I recently had, feel free to evaluate this and/or judge if this is something we really want! I won't implement this anytime soon, but maybe someone searching for a bachelor/master thesis comes to the rescue? Who knows... Cheers, Simon
Re: [julia-users] Re: @DSL Domain code
Yeah, that's fair. Its just something inside me that rebels against having code inside strings. On Sunday, November 16, 2014 9:40:07 PM UTC+2, Simon Danisch wrote: Well, the DSL macro I sketched out would allow for that. Maybe, if time comes and people think this is a good idea, this could become an extension to normal macros? But first of all, a prototype is needed anyways... How deeply the prototype will be integrated later on depends on the feedback, I guess. 2014-11-16 20:01 GMT+01:00 Johan Sigfrids johan.s...@gmail.com javascript:: Your right. That is a problem. Is there no way to extend Julia so that you can create a macro that automatically uses a custom parser? On Sunday, November 16, 2014 5:26:26 PM UTC+2, Simon Danisch wrote: No! The expression between *begin ... end* must still be valid Julia syntax, which is why you need to tell Julia, not to parse it into a Julia AST, but into some intermediate representation. @Scala begin person match { ERROR: syntax: extra token match after end of expression 2014-11-16 16:18 GMT+01:00 Johan Sigfrids johan.s...@gmail.com: Would it be impossible to simply have something like: @Scala begin person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer * case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann * case* *_* = Unknown Person } end Without having to have the multiline string thing or DSL call? On Saturday, November 15, 2014 11:55:15 PM UTC+2, Simon Danisch wrote: Hi, I was thinking about Domain Specific Languages lately from different perspectives and concluded, that we could easily make Julia an awesome home for DSL's. (Domain Specific Languages) Why would Julia benefit from a good infrastructure for DSL's? *1.)* Rapid prototyping of language concepts. I was considering Mauro's trait implementation lately, and I didn't want to use it, because it just looked a little cumbersome (no offense, its just a little difficult with only macros!). If he was able to implement the prototype first with a DSL, he could have already created a syntax prototype for it and thinks could have looked more concise and inviting. *2.)* Writing in different languages inside Julia, for example Assebler, OpenCL kernels, OpenGL shader, etc... function foo(a::Float32, b::Float32) @DSL Assembler(a, b) pushRBP mov RBP, RSP vaddss XMM0, XMM0, XMM1 pop RBP ret end I know you might ask, why this is a good idea, as LLVM should be tuned, to emit the best native code. But lets assume that it doesn't! Then, a person can already prototype how the code emitted by LLVM should look like, and another person, who probably knows LLVM a lot better, can do the appropriate changes to LLVM/Julia. *3.)* Classic use cases like a specialized UI languages, first order logic, other mathematical constructs, scripting, etc... *4.)* Stealing nice syntactic concepts from other language, to see if they give Julia any advantages, without a big hassle. function isspecificperson(person) @DSL Scala person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer *case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann *case* *_* = Unknown Person } *Possible disadvantages: * If widely used, code will get harder to read, as you need to learn all the crazy DSL's users are creating. But this will happen anyways, and it's probably better to do it in an orderly fashion than ;) *Implementation Sketch:* macro DSL(name, text) tokens = *dsltokenizer*(DSLTokens{name}(), text)::DSLTokens{name} dslast = *generate_ast*(tokens)::AST{name} return *dsl*(dslast)::AST{:Julia} end # Default implementations: *dsltokenizer*(::DSLTokens, text) = ... # Default probably with Jake Bolewkis Lexer?! *generate_ast*(text::DSLTokens) = ... *dsl*(ast::AST) = ... #Depending on the complexity of your DSL, overwrite any of these stages, to implement your own DSL, otherwise use default It seems like Jake Bolewski has already implemented a lot to make this work. Probably it would be nice to integrate OpenCL kernel code like this ;) My hope would also be, to pair this with meta data on the different stages, to make it very easy to supply correct syntax highlighting/correction for the different DSL's. Creating the ast and tokenizing things otherwise needs to be done twice, one time for the system and one time for an IDE. Or are there currently simple ways in Julia, to determine where tokens are in a string, what scopes there are and what kind of attribute println is in println(1234) ? I haven't found them yet. Most of the things you would currently need to implement for this, will be redundant to parse and are then volatile to changes in the language. Well these are just some thoughts I recently had, feel free to evaluate this and/or
Re: [julia-users] Re: @DSL Domain code
You'd need some sort of special delimiters anyway, and is pretty good because it is unlikely to occur in a DSL. It depends on how you look at it, but I'd actually argue that a string macro isn't more like a string than the rest of the code file is a string. Good editors will have a much better chance of providing correct syntax highlighting inside Scalacode here than in a block without clear delimiters. kl. 21:19:49 UTC+1 søndag 16. november 2014 skrev Johan Sigfrids følgende: Yeah, that's fair. Its just something inside me that rebels against having code inside strings. On Sunday, November 16, 2014 9:40:07 PM UTC+2, Simon Danisch wrote: Well, the DSL macro I sketched out would allow for that. Maybe, if time comes and people think this is a good idea, this could become an extension to normal macros? But first of all, a prototype is needed anyways... How deeply the prototype will be integrated later on depends on the feedback, I guess. 2014-11-16 20:01 GMT+01:00 Johan Sigfrids johan.s...@gmail.com: Your right. That is a problem. Is there no way to extend Julia so that you can create a macro that automatically uses a custom parser? On Sunday, November 16, 2014 5:26:26 PM UTC+2, Simon Danisch wrote: No! The expression between *begin ... end* must still be valid Julia syntax, which is why you need to tell Julia, not to parse it into a Julia AST, but into some intermediate representation. @Scala begin person match { ERROR: syntax: extra token match after end of expression 2014-11-16 16:18 GMT+01:00 Johan Sigfrids johan.s...@gmail.com: Would it be impossible to simply have something like: @Scala begin person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer * case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann * case* *_* = Unknown Person } end Without having to have the multiline string thing or DSL call? On Saturday, November 15, 2014 11:55:15 PM UTC+2, Simon Danisch wrote: Hi, I was thinking about Domain Specific Languages lately from different perspectives and concluded, that we could easily make Julia an awesome home for DSL's. (Domain Specific Languages) Why would Julia benefit from a good infrastructure for DSL's? *1.)* Rapid prototyping of language concepts. I was considering Mauro's trait implementation lately, and I didn't want to use it, because it just looked a little cumbersome (no offense, its just a little difficult with only macros!). If he was able to implement the prototype first with a DSL, he could have already created a syntax prototype for it and thinks could have looked more concise and inviting. *2.)* Writing in different languages inside Julia, for example Assebler, OpenCL kernels, OpenGL shader, etc... function foo(a::Float32, b::Float32) @DSL Assembler(a, b) pushRBP mov RBP, RSP vaddss XMM0, XMM0, XMM1 pop RBP ret end I know you might ask, why this is a good idea, as LLVM should be tuned, to emit the best native code. But lets assume that it doesn't! Then, a person can already prototype how the code emitted by LLVM should look like, and another person, who probably knows LLVM a lot better, can do the appropriate changes to LLVM/Julia. *3.)* Classic use cases like a specialized UI languages, first order logic, other mathematical constructs, scripting, etc... *4.)* Stealing nice syntactic concepts from other language, to see if they give Julia any advantages, without a big hassle. function isspecificperson(person) @DSL Scala person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer *case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann *case* *_* = Unknown Person } *Possible disadvantages: * If widely used, code will get harder to read, as you need to learn all the crazy DSL's users are creating. But this will happen anyways, and it's probably better to do it in an orderly fashion than ;) *Implementation Sketch:* macro DSL(name, text) tokens = *dsltokenizer*(DSLTokens{name}(), text)::DSLTokens{name} dslast = *generate_ast*(tokens)::AST{name} return *dsl*(dslast)::AST{:Julia} end # Default implementations: *dsltokenizer*(::DSLTokens, text) = ... # Default probably with Jake Bolewkis Lexer?! *generate_ast*(text::DSLTokens) = ... *dsl*(ast::AST) = ... #Depending on the complexity of your DSL, overwrite any of these stages, to implement your own DSL, otherwise use default It seems like Jake Bolewski has already implemented a lot to make this work. Probably it would be nice to integrate OpenCL kernel code like this ;) My hope would also be, to pair this with meta data on the different stages, to make it very easy to supply correct syntax highlighting/correction for the different DSL's. Creating the ast and tokenizing things otherwise needs to be done twice,
Re: [julia-users] Re: @DSL Domain code
Hehe, yeah I'm not a big fan either. But I've pinned the awkwardness down to the missing syntax highlighting and weird argument passing. Syntax could come for free with a good parser design, but argument passing is still a little bit sketchy. But I'm pretty sure, that we can find something satisfying. How Keno calls C++ functions and passes arguments in Cxx.jl doesn't look very awkward ;) 2014-11-16 21:19 GMT+01:00 Johan Sigfrids johan.sigfr...@gmail.com: Yeah, that's fair. Its just something inside me that rebels against having code inside strings. On Sunday, November 16, 2014 9:40:07 PM UTC+2, Simon Danisch wrote: Well, the DSL macro I sketched out would allow for that. Maybe, if time comes and people think this is a good idea, this could become an extension to normal macros? But first of all, a prototype is needed anyways... How deeply the prototype will be integrated later on depends on the feedback, I guess. 2014-11-16 20:01 GMT+01:00 Johan Sigfrids johan.s...@gmail.com: Your right. That is a problem. Is there no way to extend Julia so that you can create a macro that automatically uses a custom parser? On Sunday, November 16, 2014 5:26:26 PM UTC+2, Simon Danisch wrote: No! The expression between *begin ... end* must still be valid Julia syntax, which is why you need to tell Julia, not to parse it into a Julia AST, but into some intermediate representation. @Scala begin person match { ERROR: syntax: extra token match after end of expression 2014-11-16 16:18 GMT+01:00 Johan Sigfrids johan.s...@gmail.com: Would it be impossible to simply have something like: @Scala begin person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer * case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann * case* *_* = Unknown Person } end Without having to have the multiline string thing or DSL call? On Saturday, November 15, 2014 11:55:15 PM UTC+2, Simon Danisch wrote: Hi, I was thinking about Domain Specific Languages lately from different perspectives and concluded, that we could easily make Julia an awesome home for DSL's. (Domain Specific Languages) Why would Julia benefit from a good infrastructure for DSL's? *1.)* Rapid prototyping of language concepts. I was considering Mauro's trait implementation lately, and I didn't want to use it, because it just looked a little cumbersome (no offense, its just a little difficult with only macros!). If he was able to implement the prototype first with a DSL, he could have already created a syntax prototype for it and thinks could have looked more concise and inviting. *2.)* Writing in different languages inside Julia, for example Assebler, OpenCL kernels, OpenGL shader, etc... function foo(a::Float32, b::Float32) @DSL Assembler(a, b) pushRBP mov RBP, RSP vaddss XMM0, XMM0, XMM1 pop RBP ret end I know you might ask, why this is a good idea, as LLVM should be tuned, to emit the best native code. But lets assume that it doesn't! Then, a person can already prototype how the code emitted by LLVM should look like, and another person, who probably knows LLVM a lot better, can do the appropriate changes to LLVM/Julia. *3.)* Classic use cases like a specialized UI languages, first order logic, other mathematical constructs, scripting, etc... *4.)* Stealing nice syntactic concepts from other language, to see if they give Julia any advantages, without a big hassle. function isspecificperson(person) @DSL Scala person *match* { *case* Person(Hans,Meyer,7) = Found: Hans Meyer *case* Person(Heinz,Mustermann,28) = Found: Heinz Mustermann *case* *_* = Unknown Person } *Possible disadvantages: * If widely used, code will get harder to read, as you need to learn all the crazy DSL's users are creating. But this will happen anyways, and it's probably better to do it in an orderly fashion than ;) *Implementation Sketch:* macro DSL(name, text) tokens = *dsltokenizer*(DSLTokens{name}(), text)::DSLTokens{name} dslast = *generate_ast*(tokens)::AST{name} return *dsl*(dslast)::AST{:Julia} end # Default implementations: *dsltokenizer*(::DSLTokens, text) = ... # Default probably with Jake Bolewkis Lexer?! *generate_ast*(text::DSLTokens) = ... *dsl*(ast::AST) = ... #Depending on the complexity of your DSL, overwrite any of these stages, to implement your own DSL, otherwise use default It seems like Jake Bolewski has already implemented a lot to make this work. Probably it would be nice to integrate OpenCL kernel code like this ;) My hope would also be, to pair this with meta data on the different stages, to make it very easy to supply correct syntax highlighting/correction for the different DSL's. Creating the ast and tokenizing things otherwise needs to be done twice, one time for the system and one time for an IDE. Or are there
Re: [julia-users] Re: @DSL Domain code
Its just something inside me that rebels against having code inside strings. Yea, I don't like it either. If feels half-done. New token? @@