Re: New syntax for string mixins
"Don" wrote:
> It's not that complicated. Once you can load the library and call *one*
> function in it, the problem is solved.
> But at a deeper level, I'm not sure what you'd hope to achieve with this.
> I mean (excluding some not-yet implemented features), CTFE allows you to
> execute any pure + safe function which you have source code for.
>
> So, if you had such a plugin, it could do things like make database
> queries. Everything else you can do already.
May be you are right, I don't know. So yes - adding plugins is questionable
feature now. I thought on it in general, and found several things they are
suitable for, not sure if so useful:
(1) To call external tools, e.g. SQL-syntax validator, DSL code parser ;
(2) To load and handle resources at CT, e.g.
(*) load XML, JSON, etc. generated by IDE GUI and convert it to D code;
(*) load resources as handled data in inner program format, like: auto
imgData = mixin (loadJPEG ("./img/photo1.jpeg")) ; (that seems useful)
(3) make impure CTFE functions, e.g. one macro configures the state, and
some/all other depend on it (like optimisation options, hardware resources,
etc.) or cache result from call to call like fibonacci numbers.
(4) Workaround CTFE limitations if really needed (questionable, as you
pointed above).
Re: New syntax for string mixins
Alex_Dovhal wrote:
"Don" wrote:
In order for CTFE code to call pre-compiled code, three things are
required:
(1) the compiler needs to be able to find the file (.obj/.lib/shared
library) containing the compiled code;
(2) the compiler needs to be able to load the module and call it. This
requires some form of dynamic linking.
(3) We need a marshalling step, to convert from compiler literal to
compiled data, and back.
Step (3) is straightforward. The challenge is step(2), although note that
it's a general "allow the compiler to load a plugin" problem, and doesn't
have much to do with CTFE.
I thought it over, and got:
(1) can be solved by adding compiler option: --macro-libs=, so
the compiler knows in which dynamic libraries to search for the plugins ;
(2) plugin library functions should be *stdcall*, so C++ compiler can load
them. That library should implement function like that:
PluginInfo* getFunctionsInfo () ;
where:
typedef struct _PluginInfo{
struct _PluginInfo * nextPlugin ;
char* fcnName ; // short name of a function, e.g. "fcn"
char* params ; // param list, e.g. "(int, float, char*)"
char* mangledName;//name of funct. in the library, e.g. "_...@12"
char* returnType ; // e.g. "int"
bool isNothrow ; // if useful?
bool isPure ;// ditto
//... etc ...
} PluginInfo ;
And also should implement all the functions, info about which is returned by
getFunctionsInfo().
D compiler calls getFunctionsInfo from each plugin library - and then loads
all the implemented functions with full info about them.
If one plugin function name found in two or more libraries - D throws
compiler error.
Not a perfect solution, but at least straightforward one and solves given
problem.
One more note: this approach makes CTFE functions calling such plugins
uncallable at runtime, which IMO is OK, as one should not call it in any
case.
Is that suitable?
It's not that complicated. Once you can load the library and call *one*
function in it, the problem is solved.
But at a deeper level, I'm not sure what you'd hope to achieve with this.
I mean (excluding some not-yet implemented features), CTFE allows you to
execute any pure + safe function which you have source code for.
So, if you had such a plugin, it could do things like make database
queries. Everything else you can do already.
Re: New syntax for string mixins
"Don" wrote:
> In order for CTFE code to call pre-compiled code, three things are
> required:
> (1) the compiler needs to be able to find the file (.obj/.lib/shared
> library) containing the compiled code;
> (2) the compiler needs to be able to load the module and call it. This
> requires some form of dynamic linking.
> (3) We need a marshalling step, to convert from compiler literal to
> compiled data, and back.
>
>
> Step (3) is straightforward. The challenge is step(2), although note that
> it's a general "allow the compiler to load a plugin" problem, and doesn't
> have much to do with CTFE.
I thought it over, and got:
(1) can be solved by adding compiler option: --macro-libs=, so
the compiler knows in which dynamic libraries to search for the plugins ;
(2) plugin library functions should be *stdcall*, so C++ compiler can load
them. That library should implement function like that:
PluginInfo* getFunctionsInfo () ;
where:
typedef struct _PluginInfo{
struct _PluginInfo * nextPlugin ;
char* fcnName ; // short name of a function, e.g. "fcn"
char* params ; // param list, e.g. "(int, float, char*)"
char* mangledName;//name of funct. in the library, e.g. "_...@12"
char* returnType ; // e.g. "int"
bool isNothrow ; // if useful?
bool isPure ;// ditto
//... etc ...
} PluginInfo ;
And also should implement all the functions, info about which is returned by
getFunctionsInfo().
D compiler calls getFunctionsInfo from each plugin library - and then loads
all the implemented functions with full info about them.
If one plugin function name found in two or more libraries - D throws
compiler error.
Not a perfect solution, but at least straightforward one and solves given
problem.
One more note: this approach makes CTFE functions calling such plugins
uncallable at runtime, which IMO is OK, as one should not call it in any
case.
Is that suitable?
Re: New syntax for string mixins
"Don" wrote: > In order for CTFE code to call pre-compiled code, three things are > required: > (1) the compiler needs to be able to find the file (.obj/.lib/shared > library) containing the compiled code; > (2) the compiler needs to be able to load the module and call it. This > requires some form of dynamic linking. > (3) We need a marshalling step, to convert from compiler literal to > compiled data, and back. > > > Step (3) is straightforward. The challenge is step(2), although note that > it's a general "allow the compiler to load a plugin" problem, and doesn't > have much to do with CTFE. Understand. So, it should be dynamic loaded, compiler should know which D library to load for used function and this function's name mangling, also then phobos should be dynamic library to call it's functions in macro. This is non trivial stuff, and compiler itselt is written in C++ so this plugin architecture should be working in C++ too. Also when cross-compile it's neeaded compiler for both X and Y architectures or two compilers, communicating among them. So that compiler for Y when finds macro should call compiler X and dynamically load to itself produced function. OK, IMO it's too complex and experimental to be of any priority in nearest future.
Re: New syntax for string mixins
Alex_Dovhal wrote: "Don" wrote: I don't think it's quite the same. In a makefile, every executable is listed, and so you can have some degree of control over it. But in this scenario, the compiler is making calls to arbitrary shared libraries with arbitrary parameters. It means the compiler cannot be trusted *at all*. You are right only partially - it's unsafe for browser language where code is taken from untrusted source. But this feature gives so much power to the macro sysrem - that I think is worth considering it. IMO, usually compiled code is run just after compilation (with the same prermissions as compiler) - so compiled code can make dangerous things and can't be trusted at all, but no one is worry about that. Yes compiler can't be *trusted* with this features, but if one knows what he is doing, why to prevent him - add option --enable-ctfe-DANGEROUS-features to allow potentially dangerous features then it wouldn't be so unexpected. Are those features hard to add to the current implementation? In order for CTFE code to call pre-compiled code, three things are required: (1) the compiler needs to be able to find the file (.obj/.lib/shared library) containing the compiled code; (2) the compiler needs to be able to load the module and call it. This requires some form of dynamic linking. (3) We need a marshalling step, to convert from compiler literal to compiled data, and back. Step (3) is straightforward. The challenge is step(2), although note that it's a general "allow the compiler to load a plugin" problem, and doesn't have much to do with CTFE.
Re: New syntax for string mixins
VladD2 wrote: Don Wrote: I think VladD2 is right: You need to keep track of both "current" system and "target" system. Unfortunately, there is some information about the "target" system the compile-time code wouldn't be able discern without giving it the ability to run code (RPC? Virtualization? Really, really good emulator?) on the target system, but then again, that's a limitation with any cross-compiling scenario. Note that for this to work at all, the compiler needs to be able to generate exectuable code for platform X as well as for Y -- that is, it needs to include two back-ends. If the macros have been compiled and are in binary (executable) form, the compiler must only be able to generate code for platform X, Yes, but it's not a compiler for platform X! It's only a compiler for platform Y. and run macros (execute code from DLL). This is exactly what makes Nemerle compiler. The .NET system always has a compiler for the platform it's running on. That's not necessarily true for D compilers. In this case, compiling of the same macros looks like any other compilation process (on the platform X for the platform Y). I don't think it's quite the same. In a makefile, every executable is listed, and so you can have some degree of control over it. Trust to rmdir ... lol! And what about NAnt or MSBuild which can have binary extensions? I think, you are completely wrong. But in this scenario, the compiler is making calls to arbitrary shared libraries with arbitrary parameters. It means the compiler cannot be trusted *at all*. The experience of Lisp (50 years!) and Nemerel (about 6 years) shows that the ability to access any library - is not a problem. I don't think Nemerle has been sufficiently widely used, to be able to draw strong conclusions from it. The Lisp argument is strong though. > This is a huge advantage. And limit the possibility of a macro, you can simply forbidding them to use some libraries. I hope you're right, because it's indeed a powerful feature. But I'd want to hear the opinion of a security expert. In particular, if it can be shown that it's exactly the same as Lisp, I would be convinced.
Re: New syntax for string mixins
"Don" wrote: > I don't think it's quite the same. In a makefile, every executable is > listed, and so you can have some degree of control over it. But in this > scenario, the compiler is making calls to arbitrary shared libraries with > arbitrary parameters. > It means the compiler cannot be trusted *at all*. You are right only partially - it's unsafe for browser language where code is taken from untrusted source. But this feature gives so much power to the macro sysrem - that I think is worth considering it. IMO, usually compiled code is run just after compilation (with the same prermissions as compiler) - so compiled code can make dangerous things and can't be trusted at all, but no one is worry about that. Yes compiler can't be *trusted* with this features, but if one knows what he is doing, why to prevent him - add option --enable-ctfe-DANGEROUS-features to allow potentially dangerous features then it wouldn't be so unexpected. Are those features hard to add to the current implementation?
Re: New syntax for string mixins
Nick Sabalausky Wrote:
>
> The problem with that is, what if you're generating target-platform-specific
> code at compile-time? You'd be generating code for the wrong platform. I
> think VladD2 is right: You need to keep track of both "current" system and
> "target" system. Unfortunately, there is some information about the "target"
> system the compile-time code wouldn't be able discern without giving it the
> ability to run code (RPC? Virtualization? Really, really good emulator?) on
> the target system, but then again, that's a limitation with any
> cross-compiling scenario.
>
>
I don't understand this. If you're talking about version blocks, obviously all
the current D compile time features are incompatible with the macro system.
They're also redundant cause you can (and should) simply write instead:
macro (version) {
if (version == Version.Linux) { ...}
}
Also, the macro dll is compiled for some platform Z and will only be loadable
by a compiler that also runs on platform Z. I don't think that macros need to
be cross-compiled for our (X,Y) scenario, they need to evaluate a
TargetPlatform parameter at their run-time which the cross-compiler will
provide (much like current D have predefined version identifiers defined by the
compiler)
Re: New syntax for string mixins
Don Wrote: > > I > > think VladD2 is right: You need to keep track of both "current" system and > > "target" system. Unfortunately, there is some information about the > > "target" > > system the compile-time code wouldn't be able discern without giving it the > > ability to run code (RPC? Virtualization? Really, really good emulator?) on > > the target system, but then again, that's a limitation with any > > cross-compiling scenario. > > Note that for this to work at all, the compiler needs to be able to > generate exectuable code for platform X as well as for Y -- that is, it > needs to include two back-ends. If the macros have been compiled and are in binary (executable) form, the compiler must only be able to generate code for platform X, and run macros (execute code from DLL). This is exactly what makes Nemerle compiler. In this case, compiling of the same macros looks like any other compilation process (on the platform X for the platform Y). > I don't think it's quite the same. In a makefile, every executable is > listed, and so you can have some degree of control over it. Trust to rmdir ... lol! And what about NAnt or MSBuild which can have binary extensions? I think, you are completely wrong. > But in this > scenario, the compiler is making calls to arbitrary shared libraries > with arbitrary parameters. > It means the compiler cannot be trusted *at all*. The experience of Lisp (50 years!) and Nemerel (about 6 years) shows that the ability to access any library - is not a problem. This is a huge advantage. And limit the possibility of a macro, you can simply forbidding them to use some libraries.
Re: New syntax for string mixins
"Don" wrote in message news:iej7eu$b1...@digitalmars.com... > Nick Sabalausky wrote: >> "foobar" wrote in message >> news:ieijt6$21m...@digitalmars.com... >>> I don't see how there needs to be different code to accomplish the above >>> use case. You have a function that tests the hardware it's being run on. >>> When this function is called in the compiler context it would return X, >>> when it's called from the target executable it returns Y. >>> >> >> The problem with that is, what if you're generating >> target-platform-specific code at compile-time? You'd be generating code >> for the wrong platform. I think VladD2 is right: You need to keep track >> of both "current" system and "target" system. Unfortunately, there is >> some information about the "target" system the compile-time code wouldn't >> be able discern without giving it the ability to run code (RPC? >> Virtualization? Really, really good emulator?) on the target system, but >> then again, that's a limitation with any cross-compiling scenario. > > Note that for this to work at all, the compiler needs to be able to > generate exectuable code for platform X as well as for Y -- that is, it > needs to include two back-ends. > But if the compiler doesn't have both backends then the whole question of "how does the user's compile-time code get handled if it's being cross-compiled?" becomes irrelevent - you'd have to compile it on the target plaform anyway, so X == Y. Or am I missing your point? I'm also scared of the implications of allowing arbitrary code execution during compilation. Make a typo in your program, and then compilation may wipe files from your hard disk, or corrupt an external database, etc... On some platforms you may be able to sandbox it, but since it's running as part of the compilation process, rather than with the permissions it will eventually have, it just seems like a security nightmare. >> >> That's an interesting point. OTOH, it's common for software to come with >> its own build script or makefile, and those can certainly do literally >> anything like you describe above - but I haven't seen that as being a >> real problem. > > I don't think it's quite the same. In a makefile, every executable is > listed, and so you can have some degree of control over it. But in this > scenario, the compiler is making calls to arbitrary shared libraries with > arbitrary parameters. > It means the compiler cannot be trusted *at all*. I suppose that's a reasonable point.
Re: New syntax for string mixins
Nick Sabalausky wrote: "foobar" wrote in message news:ieijt6$21m...@digitalmars.com... Don Wrote: VladD2 wrote: Don Wrote: Suppose the pre-compiled code, when run, asks what CPU it's on. What's the answer? Is it X or Y? Current: X Target: Y Macro - a plugin to the compiler. It works on the same platform as the compiler, but generates code through the API which abstracts the macro from the target platform. If you need generate platform specific code, you should worry about it in macro logic. In any case macro is a meta-programm wich generate or/and transform code. Yes. But in D there's no distinction between code which is destined for a macro, versus any other function. You can call a function once at compile time, and the same function at compile time. I think you mean "You can call a function once at compile time, and the same function at **runtime**." My understanding of Nemerle (which is quite likely to be wrong!) is that at least some functions are callable only at compile-time. I'd be surprised. I would think that all you would have to do to use the same Nemerle function at both runtime and compile-time would be to include its module in both the "compile the compiler-plugin step" and in the "load compiler-plugins and compile the app" step. I don't see how there needs to be different code to accomplish the above use case. You have a function that tests the hardware it's being run on. When this function is called in the compiler context it would return X, when it's called from the target executable it returns Y. The problem with that is, what if you're generating target-platform-specific code at compile-time? You'd be generating code for the wrong platform. I think VladD2 is right: You need to keep track of both "current" system and "target" system. Unfortunately, there is some information about the "target" system the compile-time code wouldn't be able discern without giving it the ability to run code (RPC? Virtualization? Really, really good emulator?) on the target system, but then again, that's a limitation with any cross-compiling scenario. Note that for this to work at all, the compiler needs to be able to generate exectuable code for platform X as well as for Y -- that is, it needs to include two back-ends. I'm also scared of the implications of allowing arbitrary code execution during compilation. Make a typo in your program, and then compilation may wipe files from your hard disk, or corrupt an external database, etc... On some platforms you may be able to sandbox it, but since it's running as part of the compilation process, rather than with the permissions it will eventually have, it just seems like a security nightmare. That's an interesting point. OTOH, it's common for software to come with its own build script or makefile, and those can certainly do literally anything like you describe above - but I haven't seen that as being a real problem. I don't think it's quite the same. In a makefile, every executable is listed, and so you can have some degree of control over it. But in this scenario, the compiler is making calls to arbitrary shared libraries with arbitrary parameters. It means the compiler cannot be trusted *at all*.
Re: New syntax for string mixins
"foobar" wrote in message news:ieijt6$21m...@digitalmars.com... > Don Wrote: > >> VladD2 wrote: >> > Don Wrote: >> >> Suppose the pre-compiled code, when run, asks what CPU it's on. What's >> >> the answer? Is it X or Y? >> > >> > Current: X >> > Target: Y >> > >> > Macro - a plugin to the compiler. It works on the same platform as the >> > compiler, but generates code through the API which abstracts the macro >> > from the target platform. If you need generate platform specific code, >> > you should worry about it in macro logic. >> > In any case macro is a meta-programm wich generate or/and transform >> > code. >> >> Yes. But in D there's no distinction between code which is destined for >> a macro, versus any other function. You can call a function once at >> compile time, and the same function at compile time. I think you mean "You can call a function once at compile time, and the same function at **runtime**." >> My understanding of >> Nemerle (which is quite likely to be wrong!) is that at least some >> functions are callable only at compile-time. >> I'd be surprised. I would think that all you would have to do to use the same Nemerle function at both runtime and compile-time would be to include its module in both the "compile the compiler-plugin step" and in the "load compiler-plugins and compile the app" step. > > I don't see how there needs to be different code to accomplish the above > use case. You have a function that tests the hardware it's being run on. > When this function is called in the compiler context it would return X, > when it's called from the target executable it returns Y. > The problem with that is, what if you're generating target-platform-specific code at compile-time? You'd be generating code for the wrong platform. I think VladD2 is right: You need to keep track of both "current" system and "target" system. Unfortunately, there is some information about the "target" system the compile-time code wouldn't be able discern without giving it the ability to run code (RPC? Virtualization? Really, really good emulator?) on the target system, but then again, that's a limitation with any cross-compiling scenario. >> I'm also scared of the implications of allowing arbitrary code execution >> during compilation. Make a typo in your program, and then compilation >> may wipe files from your hard disk, or corrupt an external database, >> etc... On some platforms you may be able to sandbox it, but since it's >> running as part of the compilation process, rather than with the >> permissions it will eventually have, it just seems like a security >> nightmare. >> That's an interesting point. OTOH, it's common for software to come with its own build script or makefile, and those can certainly do literally anything like you describe above - but I haven't seen that as being a real problem. > > This is a void argument since templates are Turing complete. > It's *already* possible to do all of the above in D at compile time, it's > just a matter of how much code is required to accomplish this. > Not true. This is a frequent misconception about Turning-completeness. Just because something is Turing complete does *not* mean it can do anything that any other Turing complete system can do (contrary to how many college profs explain it). It *does* mean that it can *calculate* anything that any other Turing complete system can calculate. But it doesn't necessarily have *access* to everything that any other Turing complete system has access to. And D's compile-time system, whether CTFE or templates, does not currently provide any way to access any I/O, launch any processes, or do any direct memory access.
Re: New syntax for string mixins
On 2010-12-17 00:34, Graham St Jack wrote: I've attached a part of how concurrency.d could look like translated to my suggested syntax. It probably contains a lot of errors because did a quick translation and I had some trouble understanding the mixins. Yes, even I couldn't understand them even a week later. Maintenance is a real problem. My initial reaction is that the proposed syntax helps a bit, but isn't really a game-changer. I will let you know if I change my mind on closer inspection. No it's basically just syntactic sugar. -- /Jacob Carlborg
Re: New syntax for string mixins
Don Wrote: > VladD2 wrote: > > Don Wrote: > >> Suppose the pre-compiled code, when run, asks what CPU it's on. What's > >> the answer? Is it X or Y? > > > > Current: X > > Target: Y > > > > Macro - a plugin to the compiler. It works on the same platform as the > > compiler, but generates code through the API which abstracts the macro from > > the target platform. If you need generate platform specific code, you > > should worry about it in macro logic. > > In any case macro is a meta-programm wich generate or/and transform code. > > Yes. But in D there's no distinction between code which is destined for > a macro, versus any other function. You can call a function once at > compile time, and the same function at compile time. My understanding of > Nemerle (which is quite likely to be wrong!) is that at least some > functions are callable only at compile-time. > I don't see how there needs to be different code to accomplish the above use case. You have a function that tests the hardware it's being run on. When this function is called in the compiler context it would return X, when it's called from the target executable it returns Y. > I'm also scared of the implications of allowing arbitrary code execution > during compilation. Make a typo in your program, and then compilation > may wipe files from your hard disk, or corrupt an external database, > etc... On some platforms you may be able to sandbox it, but since it's > running as part of the compilation process, rather than with the > permissions it will eventually have, it just seems like a security > nightmare. > This is a void argument since templates are Turing complete. It's *already* possible to do all of the above in D at compile time, it's just a matter of how much code is required to accomplish this. > All these problems would be reduced somewhat if it were only permissible > to call pure functions at compile time. Although even that has issues, > since not all compiler platforms may allow calling of shared libraries. > If we build that into the language, we'd be cutting ourselves off from > those platforms. > > I'm not certain that there are any unsurmountable problems, but these > issues make me really cautious.
Re: New syntax for string mixins
VladD2 wrote: Don Wrote: Suppose the pre-compiled code, when run, asks what CPU it's on. What's the answer? Is it X or Y? Current: X Target: Y Macro - a plugin to the compiler. It works on the same platform as the compiler, but generates code through the API which abstracts the macro from the target platform. If you need generate platform specific code, you should worry about it in macro logic. In any case macro is a meta-programm wich generate or/and transform code. Yes. But in D there's no distinction between code which is destined for a macro, versus any other function. You can call a function once at compile time, and the same function at compile time. My understanding of Nemerle (which is quite likely to be wrong!) is that at least some functions are callable only at compile-time. I'm also scared of the implications of allowing arbitrary code execution during compilation. Make a typo in your program, and then compilation may wipe files from your hard disk, or corrupt an external database, etc... On some platforms you may be able to sandbox it, but since it's running as part of the compilation process, rather than with the permissions it will eventually have, it just seems like a security nightmare. All these problems would be reduced somewhat if it were only permissible to call pure functions at compile time. Although even that has issues, since not all compiler platforms may allow calling of shared libraries. If we build that into the language, we'd be cutting ourselves off from those platforms. I'm not certain that there are any unsurmountable problems, but these issues make me really cautious.
Re: New syntax for string mixins
Don Wrote: > > You missed: pattern matching, > > Yes, you've got me there. I've assumed that pattern matching, while a > major feature, is not fundamental to the Nemerle macro-system, but I may > be mistaken. > > One obvious difference which *is* fundamental is that Nemerle macros > allow new syntax. > > Am I correct in thinking that Nemerle always requires complete ASTs? > That is, given a name "x", can you access variables "x1", "x2", "x3" ? > > > quasi-quotation, > > No, that's present in D. It's the primary reason I say that the > differences are mostly syntactic, since I see this as THE fundamental > feature. You need to be able to reenter the macro system after you have > left it. Once you can do that, you can do pretty much anything. You not understand... pattern matching and quasi-quotation tightly bound. The quasi-quotation produce instances of Algebraic Data Type (AlgTD) which can be decomposed (and matched) by pattern matching (PM). We use PM to "read" code of a project. For example, see C# grammar: http://code.google.com/p/nemerle/source/browse/nemerle/trunk/snippets/csharp-parser/CSharpParser/Parser.n?r=9436 which made by PegGrammar macro: http://code.google.com/p/nemerle/source/browse/nemerle/trunk/snippets/peg-parser/Nemerle.Peg.Macros?r=9436 Or see our foreach macro: http://code.google.com/p/nemerle/source/browse/nemerle/trunk/macros/core.n?r=9436#508 > > running of fully functional Nemerle > > code in compile time, > > Yes, but that's a different issue. In D, running code at compile time is > regarded as an aspect of constant-folding, and is not restricted to > macros. The constant-folding is too limited solution. The power of of macros, including the ability to use are any libraries and any data sources. And ... you use enterpretation... slow way. > > access to compiler API in macros > > In practice, has to be a library, right? Otherwise the compiler > internals would be exposed? (This is an issue we're struggling with). Yes. The Nemerle compiler is library. :) Unfortunately Nemerle API is not pretty clear. But I believe it's a right way. P.S. Sorry for my (Russian) English. :)
Re: New syntax for string mixins
Don Wrote: > Suppose the pre-compiled code, when run, asks what CPU it's on. What's > the answer? Is it X or Y? Current: X Target: Y Macro - a plugin to the compiler. It works on the same platform as the compiler, but generates code through the API which abstracts the macro from the target platform. If you need generate platform specific code, you should worry about it in macro logic. In any case macro is a meta-programm wich generate or/and transform code.
Re: New syntax for string mixins
foobar wrote: Don Wrote: Another one of the big differences is that D doesn't allow compile-time code to call external functions. Although it could certainly be done, it raises some big issues. Eg, we cannot assume that the target CPU is the same as the one we're running on. With a JIT compiler, you don't have that problem. Don, can you please elaborate on this point? Here's my understanding: The D compiler is run once to both 'execute' compile time code which you refer above as constant folding AND to generate the binary to execute at run-time. Nemerle separates this into two distinct steps: 1. you compile regular code inside macro definitions into a compiler plugin. 2. when compiling the intended run-time code you need to load the compiled macros from step 1 above on the command line of the compiler. Since you're talking above about cross-compilation and let's say we run the compiler on X and compile for Y, I see no problem to load precompiled macros for X in order to compile the code for Y. The only limit as far as I can see is that it won't be possible to load macros compiled by the such a cross compiler on X unless as you say JIT is employed. Perhaps it makes sense to disable macro definition in a cross-compilation scenario and only allow usage. Suppose the pre-compiled code, when run, asks what CPU it's on. What's the answer? Is it X or Y?
Re: New syntax for string mixins
Don Wrote: > VladD2 wrote: > > Hi, > > > > I'm one of Nemerle developers. > > Cool! > > >> From an implementation point of view, the differences between Nemerle > >> macros and string mixins are mostly syntactic. > > > > You are deeply mistaken! > > I'd advise to study the Nemerle macro-system. > > > > You missed: pattern matching, > > Yes, you've got me there. I've assumed that pattern matching, while a > major feature, is not fundamental to the Nemerle macro-system, but I may > be mistaken. > > One obvious difference which *is* fundamental is that Nemerle macros > allow new syntax. > > Am I correct in thinking that Nemerle always requires complete ASTs? > That is, given a name "x", can you access variables "x1", "x2", "x3" ? > > > quasi-quotation, > > No, that's present in D. It's the primary reason I say that the > differences are mostly syntactic, since I see this as THE fundamental > feature. You need to be able to reenter the macro system after you have > left it. Once you can do that, you can do pretty much anything. > > BTW if you argued that D's current syntax is quite horrible, I couldn't > disagree with you. > > > running of fully functional Nemerle > > code in compile time, > > Yes, but that's a different issue. In D, running code at compile time is > regarded as an aspect of constant-folding, and is not restricted to > macros. > > > access to compiler API in macros > > In practice, has to be a library, right? Otherwise the compiler > internals would be exposed? (This is an issue we're struggling with). > > Another one of the big differences is that D doesn't allow compile-time > code to call external functions. Although it could certainly be done, it > raises some big issues. Eg, we cannot assume that the target CPU is the > same as the one we're running on. With a JIT compiler, you don't have > that problem. Don, can you please elaborate on this point? Here's my understanding: The D compiler is run once to both 'execute' compile time code which you refer above as constant folding AND to generate the binary to execute at run-time. Nemerle separates this into two distinct steps: 1. you compile regular code inside macro definitions into a compiler plugin. 2. when compiling the intended run-time code you need to load the compiled macros from step 1 above on the command line of the compiler. Since you're talking above about cross-compilation and let's say we run the compiler on X and compile for Y, I see no problem to load precompiled macros for X in order to compile the code for Y. The only limit as far as I can see is that it won't be possible to load macros compiled by the such a cross compiler on X unless as you say JIT is employed. Perhaps it makes sense to disable macro definition in a cross-compilation scenario and only allow usage. Is there something else that I'm missing here? > > > and IntelliSense support. > > I did mention that... > > >> The one thing about them that I find really impressive is the IDE > >> integration, especially that they got syntax highlighting to work. > >> I don't know they've done that. > > > > "syntax highlighting" ? :) > > We have full IntelliSense support: syntax highlighting, code completion, > > macro > > expansion (in tooltips). > > I meant Intellisense, not just syntax highlighting. Though note that you > can't do syntax highlighting *perfectly* unless the IDE understands the > code. > > > We simply use the Nemerle compiler to achive that. > > Doesn't leave me any less impressed.
Re: New syntax for string mixins
Nick Sabalausky wrote: > "Jacob Carlborg" wrote in message > news:iednio$2vj...@digitalmars.com... >> >> I can't quite visualize how the final code will look like and as you say >> it's hard without a formal definition of AST macros. But I think somehow >> it would be possible, I mean instead of combining strings one combine >> expressions/syntax. But I don't know if it would be possible to combine >> incomplete expressions. >> > > One parallel that may or may not be applicable, but might be worth > considering, is dynamically building XHTML: Using a string-template system > is generally found to work very well, but building it via a DOM > (essentially an AST) is often considered a bit of a pain. I guess the > simplest take-away would be that string-based approaches may be easier get > working well. FWIW. The newest vb.net has this feature, if I understand you correctly, via xml literals and linq expressions. It is quite convenient. http://msdn.microsoft.com/en-us/library/bb384460.aspx
Re: New syntax for string mixins
VladD2 wrote: Hi, I'm one of Nemerle developers. Cool! From an implementation point of view, the differences between Nemerle macros and string mixins are mostly syntactic. You are deeply mistaken! I'd advise to study the Nemerle macro-system. You missed: pattern matching, Yes, you've got me there. I've assumed that pattern matching, while a major feature, is not fundamental to the Nemerle macro-system, but I may be mistaken. One obvious difference which *is* fundamental is that Nemerle macros allow new syntax. Am I correct in thinking that Nemerle always requires complete ASTs? That is, given a name "x", can you access variables "x1", "x2", "x3" ? > quasi-quotation, No, that's present in D. It's the primary reason I say that the differences are mostly syntactic, since I see this as THE fundamental feature. You need to be able to reenter the macro system after you have left it. Once you can do that, you can do pretty much anything. BTW if you argued that D's current syntax is quite horrible, I couldn't disagree with you. running of fully functional Nemerle code in compile time, Yes, but that's a different issue. In D, running code at compile time is regarded as an aspect of constant-folding, and is not restricted to macros. access to compiler API in macros In practice, has to be a library, right? Otherwise the compiler internals would be exposed? (This is an issue we're struggling with). Another one of the big differences is that D doesn't allow compile-time code to call external functions. Although it could certainly be done, it raises some big issues. Eg, we cannot assume that the target CPU is the same as the one we're running on. With a JIT compiler, you don't have that problem. and IntelliSense support. I did mention that... The one thing about them that I find really impressive is the IDE integration, especially that they got syntax highlighting to work. I don't know they've done that. "syntax highlighting" ? :) We have full IntelliSense support: syntax highlighting, code completion, macro expansion (in tooltips). I meant Intellisense, not just syntax highlighting. Though note that you can't do syntax highlighting *perfectly* unless the IDE understands the code. We simply use the Nemerle compiler to achive that. Doesn't leave me any less impressed.
Re: New syntax for string mixins
Hi, I'm one of Nemerle developers. > From an implementation point of view, the differences between Nemerle > macros and string mixins are mostly syntactic. You are deeply mistaken! I'd advise to study the Nemerle macro-system. You missed: pattern matching, quasi-quotation, running of fully functional Nemerle code in compile time, access to compiler API in macros and IntelliSense support. >The one thing about them that I find really impressive is the IDE >integration, especially that they got syntax highlighting to work. >I don't know they've done that. "syntax highlighting" ? :) We have full IntelliSense support: syntax highlighting, code completion, macro expansion (in tooltips). We simply use the Nemerle compiler to achive that.
Re: New syntax for string mixins
I've attached a part of how concurrency.d could look like translated to my suggested syntax. It probably contains a lot of errors because did a quick translation and I had some trouble understanding the mixins. Yes, even I couldn't understand them even a week later. Maintenance is a real problem. My initial reaction is that the proposed syntax helps a bit, but isn't really a game-changer. I will let you know if I change my mind on closer inspection. -- Graham St Jack
Re: New syntax for string mixins
"Jacob Carlborg" wrote in message
news:iee561$v5...@digitalmars.com...
> On 2010-12-16 23:05, Jonathan M Davis wrote:
>> On Thursday, December 16, 2010 11:28:03 Jacob Carlborg wrote:
>>> On 2010-12-15 23:00, Nick Sabalausky wrote:
"Jonathan M Davis" wrote in message
news:mailman.1035.1292441722.21107.digitalmar...@puremagic.com...
> On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
>> That was my idea as well, that
>>
>> @get_set("int", "bar");
>>
>> could be translated into
>>
>> mixin(get_set("int", "bar")); just like
>>
>> just like scope statements are translated into try/catch/finally.
>
> Honestly, I don't see much gain in using @ rather than mixin(). It's a
> little
> less typing, but that's it.
It does seem like a small difference, just replacing "mixin" with "@"
and
removing one layer of parens. But I think that extra layer of parens,
minor as it seems, makes a big difference in the readability (and
"typeability") of mixin invocations. Those extra parens do get to be a
real bother, major visual noise at least to my eyes.
>>>
>>> ^^ I completely agree.
>>>
> And it precludes stuff like mixin("lhs " ~ op ~ "
> rhs") like happens all the time in overloaded operator functions.
I don't see why these shouldn't work:
@"int foo;";
return @("lhs " ~ op ~ " rhs");
At least with just the "@" part of the proposal. Maybe the delegate
thing
might make it tricker, I dunno.
>>>
>>> My idea was actually to get rid of the strings where the code to be
>>> mixed in is defined and to have a better syntax where it's used.
>>>
>>> The delegates are just a way of passing a block of code around. If you
>>> just use it in place then maybe one could do like this:
>>>
>>> @(int foo;);
>>> return @(lhs@(op)rhs);
>>
>> I would have thought that template mixins would be the thing to use when
>> you
>> didn't want to deal with strings. string mixins are extremely powerful
>> and
>> flexible, and I'd really hate to lose them. And IIRC, Kenji Hara was
>> working on a
>> module to really help make dealing with complicated string mixins easier
>> and
>> less painful. Anything that you propose is going to have to have major
>> benefits
>> over the current string mixin situation for it to stand any chance of
>> being
>> accepted.
>>
>> - Jonathan M Davis
>
> Template mixins and string mixins are used for different things. There's a
> lot of things that string mixins can do that template mixins can't. I have
> no intention what so ever to suggest something that isn't as powerful as
> string mixins, just a new syntax. If it turns out that a having a powerful
> syntax without strings isn't possibles than I'll guess we have to live
> with the strings.
>
> Don't know if you read my first post put there I wrote that it wasn't a
> real suggestion (at least not yet) I just wanted the community's thoughts
> on the idea and see if we could turn it into something useful that could
> become a real suggestion, if people where interested.
>
> The ideas I wrote in my extended suggestion, "Taking it one step further",
> I think that those can have benefits over string mixins. Basically
> allowing you to pass the whole body of a class declaration to a function,
> as a delegate, with a syntax looking like Java annotations.
>
It would seem to make sense to treat "chucks of code" the same way
regardless of whether you're passing them around, mixing them in or
instantiating them as a template.
Re: New syntax for string mixins
"Jacob Carlborg" wrote in message
news:iee4en$tt...@digitalmars.com...
> On 2010-12-16 21:35, Nick Sabalausky wrote:
>> "Jacob Carlborg" wrote in message
>> news:iedpbg$3i...@digitalmars.com...
>>> On 2010-12-15 23:00, Nick Sabalausky wrote:
I don't see why these shouldn't work:
@"int foo;";
return @("lhs " ~ op ~ " rhs");
At least with just the "@" part of the proposal. Maybe the delegate
thing
might make it tricker, I dunno.
>>>
>>> My idea was actually to get rid of the strings where the code to be
>>> mixed
>>> in is defined and to have a better syntax where it's used.
>>>
>>> The delegates are just a way of passing a block of code around. If you
>>> just use it in place then maybe one could do like this:
>>>
>>> @(int foo;);
>>> return @(lhs@(op)rhs);
>>>
>>
>> Yea, my point was just that the "@..." stuff could work either way, with
>> the
>> string-based system or with your delegate-based one.
>>
>> I don't mean to come across like I'm ignoring or against the idea of the
>> whole delegate aspect, and I understand that the main point of the OP is
>> to
>> replace the strings with delegates, but with the q{...} syntax and
>> string-templating, I'm still struggling to see a big enough benefit
>> compared
>> to the status quo. I see that using delegates instead of strings could
>> probably be made to work, but my questions are "For what benefit(s)?" and
>> "Would those benefits be sufficient to warrant the change?" I'm not
>> necessarily saying the answer is "no", but I'm unconvinced so far.
>>
>> And here's another thing: Suppose we got a Ruby/PHP-like syntax for
>> embedding code substitutions directly into a string (which would have
>> other
>> useful applications besides mixins):
>>
>> auto name = "Joe";
>> auto msg = "hello #{name}, whaddup?";
>> mixin( q{ int #{name} = 7; } );
>> Joe++;
>>
>> Would that eliminate much (or all) of the benefit of the delegate
>> approach?
>
> I guess using q{...} with string interpolation is very similar to the
> delegate approach. It just feels wrong passing around strings to represent
> code.
Well, code *is* text after all. But I know what you mean - after all, it
does have more semantic structure than just ordinary generic strings.
> I haven't though much about it but with delegates one could at lest hope
> for better help from the compiler validating the code. I don't know how
> IDEs will treat q{...} but with delegates you would get the full benefit
> of the IDE like autocompletion and similar features.
>
My editor (Programmer's Notepad 2, based off Scintilla) handles that fine.
It doesn't know anything about q{}, so it assumes it's an identifier ("q")
followed by a normal code block. And since it doesn't try to do any
grammatical/semantic validation (only lexical, and only for the purpose of
highlighting) it doesn't complain about "identifier { ... }" being invalid
or any of the indentifiers-to-be-replaced inside of it being undeclared.
But for fancier IDE's, like Eclipse with Descent or DDT, I don't know -
that's a good question.
OTOH, even with the delegate approach, I'm assuming that delegate would
still get evaluated in a different context from where it's defined (which
you'd probably want). So that might still cause some trouble with the more
intelligent IDEs trying to tell you that identifierXYZ isn't accessable from
within what it thinks is the delegate's scope.
Re: New syntax for string mixins
On 2010-12-16 23:05, Jonathan M Davis wrote:
On Thursday, December 16, 2010 11:28:03 Jacob Carlborg wrote:
On 2010-12-15 23:00, Nick Sabalausky wrote:
"Jonathan M Davis" wrote in message
news:mailman.1035.1292441722.21107.digitalmar...@puremagic.com...
On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
That was my idea as well, that
@get_set("int", "bar");
could be translated into
mixin(get_set("int", "bar")); just like
just like scope statements are translated into try/catch/finally.
Honestly, I don't see much gain in using @ rather than mixin(). It's a
little
less typing, but that's it.
It does seem like a small difference, just replacing "mixin" with "@" and
removing one layer of parens. But I think that extra layer of parens,
minor as it seems, makes a big difference in the readability (and
"typeability") of mixin invocations. Those extra parens do get to be a
real bother, major visual noise at least to my eyes.
^^ I completely agree.
And it precludes stuff like mixin("lhs " ~ op ~ "
rhs") like happens all the time in overloaded operator functions.
I don't see why these shouldn't work:
@"int foo;";
return @("lhs " ~ op ~ " rhs");
At least with just the "@" part of the proposal. Maybe the delegate thing
might make it tricker, I dunno.
My idea was actually to get rid of the strings where the code to be
mixed in is defined and to have a better syntax where it's used.
The delegates are just a way of passing a block of code around. If you
just use it in place then maybe one could do like this:
@(int foo;);
return @(lhs@(op)rhs);
I would have thought that template mixins would be the thing to use when you
didn't want to deal with strings. string mixins are extremely powerful and
flexible, and I'd really hate to lose them. And IIRC, Kenji Hara was working on
a
module to really help make dealing with complicated string mixins easier and
less painful. Anything that you propose is going to have to have major benefits
over the current string mixin situation for it to stand any chance of being
accepted.
- Jonathan M Davis
Template mixins and string mixins are used for different things. There's
a lot of things that string mixins can do that template mixins can't. I
have no intention what so ever to suggest something that isn't as
powerful as string mixins, just a new syntax. If it turns out that a
having a powerful syntax without strings isn't possibles than I'll guess
we have to live with the strings.
Don't know if you read my first post put there I wrote that it wasn't a
real suggestion (at least not yet) I just wanted the community's
thoughts on the idea and see if we could turn it into something useful
that could become a real suggestion, if people where interested.
The ideas I wrote in my extended suggestion, "Taking it one step
further", I think that those can have benefits over string mixins.
Basically allowing you to pass the whole body of a class declaration to
a function, as a delegate, with a syntax looking like Java annotations.
--
/Jacob Carlborg
Re: New syntax for string mixins
On 2010-12-16 21:35, Nick Sabalausky wrote:
"Jacob Carlborg" wrote in message
news:iedpbg$3i...@digitalmars.com...
On 2010-12-15 23:00, Nick Sabalausky wrote:
I don't see why these shouldn't work:
@"int foo;";
return @("lhs " ~ op ~ " rhs");
At least with just the "@" part of the proposal. Maybe the delegate thing
might make it tricker, I dunno.
My idea was actually to get rid of the strings where the code to be mixed
in is defined and to have a better syntax where it's used.
The delegates are just a way of passing a block of code around. If you
just use it in place then maybe one could do like this:
@(int foo;);
return @(lhs@(op)rhs);
Yea, my point was just that the "@..." stuff could work either way, with the
string-based system or with your delegate-based one.
I don't mean to come across like I'm ignoring or against the idea of the
whole delegate aspect, and I understand that the main point of the OP is to
replace the strings with delegates, but with the q{...} syntax and
string-templating, I'm still struggling to see a big enough benefit compared
to the status quo. I see that using delegates instead of strings could
probably be made to work, but my questions are "For what benefit(s)?" and
"Would those benefits be sufficient to warrant the change?" I'm not
necessarily saying the answer is "no", but I'm unconvinced so far.
And here's another thing: Suppose we got a Ruby/PHP-like syntax for
embedding code substitutions directly into a string (which would have other
useful applications besides mixins):
auto name = "Joe";
auto msg = "hello #{name}, whaddup?";
mixin( q{ int #{name} = 7; } );
Joe++;
Would that eliminate much (or all) of the benefit of the delegate approach?
I guess using q{...} with string interpolation is very similar to the
delegate approach. It just feels wrong passing around strings to
represent code. I haven't though much about it but with delegates one
could at lest hope for better help from the compiler validating the
code. I don't know how IDEs will treat q{...} but with delegates you
would get the full benefit of the IDE like autocompletion and similar
features.
--
/Jacob Carlborg
Re: New syntax for string mixins
On 2010-12-16 21:13, Nick Sabalausky wrote: "Jacob Carlborg" wrote in message news:iednio$2vj...@digitalmars.com... I can't quite visualize how the final code will look like and as you say it's hard without a formal definition of AST macros. But I think somehow it would be possible, I mean instead of combining strings one combine expressions/syntax. But I don't know if it would be possible to combine incomplete expressions. One parallel that may or may not be applicable, but might be worth considering, is dynamically building XHTML: Using a string-template system is generally found to work very well, but building it via a DOM (essentially an AST) is often considered a bit of a pain. I guess the simplest take-away would be that string-based approaches may be easier get working well. FWIW. I the case of XML I think it can be quite easy if you use the right libraries/tools. I think the easiest library I've used for building XML files is the Ruby library Bulilder, a code example using Builder can look like this: xml = Builder::XmlMarkup.new xml.person do xml.first_name "John" xml.last_name "Doe" xml.phone "5484654", :type => "mobile" end Which will generate this: John Doe 5484654 Now I don't think that this library is a DOM library, which allows you to manipulate a DOM tree, it's something simpler that just generates XML. Link: http://builder.rubyforge.org/ -- /Jacob Carlborg
Re: New syntax for string mixins
On Thursday, December 16, 2010 11:28:03 Jacob Carlborg wrote:
> On 2010-12-15 23:00, Nick Sabalausky wrote:
> > "Jonathan M Davis" wrote in message
> > news:mailman.1035.1292441722.21107.digitalmar...@puremagic.com...
> >
> >> On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
> >>> That was my idea as well, that
> >>>
> >>> @get_set("int", "bar");
> >>>
> >>> could be translated into
> >>>
> >>> mixin(get_set("int", "bar")); just like
> >>>
> >>> just like scope statements are translated into try/catch/finally.
> >>
> >> Honestly, I don't see much gain in using @ rather than mixin(). It's a
> >> little
> >> less typing, but that's it.
> >
> > It does seem like a small difference, just replacing "mixin" with "@" and
> > removing one layer of parens. But I think that extra layer of parens,
> > minor as it seems, makes a big difference in the readability (and
> > "typeability") of mixin invocations. Those extra parens do get to be a
> > real bother, major visual noise at least to my eyes.
>
> ^^ I completely agree.
>
> >> And it precludes stuff like mixin("lhs " ~ op ~ "
> >> rhs") like happens all the time in overloaded operator functions.
> >
> > I don't see why these shouldn't work:
> >
> > @"int foo;";
> > return @("lhs " ~ op ~ " rhs");
> >
> > At least with just the "@" part of the proposal. Maybe the delegate thing
> > might make it tricker, I dunno.
>
> My idea was actually to get rid of the strings where the code to be
> mixed in is defined and to have a better syntax where it's used.
>
> The delegates are just a way of passing a block of code around. If you
> just use it in place then maybe one could do like this:
>
> @(int foo;);
> return @(lhs@(op)rhs);
I would have thought that template mixins would be the thing to use when you
didn't want to deal with strings. string mixins are extremely powerful and
flexible, and I'd really hate to lose them. And IIRC, Kenji Hara was working on
a
module to really help make dealing with complicated string mixins easier and
less painful. Anything that you propose is going to have to have major benefits
over the current string mixin situation for it to stand any chance of being
accepted.
- Jonathan M Davis
Re: New syntax for string mixins
"Jacob Carlborg" wrote in message
news:iedpbg$3i...@digitalmars.com...
> On 2010-12-15 23:00, Nick Sabalausky wrote:
>>
>> I don't see why these shouldn't work:
>>
>> @"int foo;";
>> return @("lhs " ~ op ~ " rhs");
>>
>> At least with just the "@" part of the proposal. Maybe the delegate thing
>> might make it tricker, I dunno.
>
> My idea was actually to get rid of the strings where the code to be mixed
> in is defined and to have a better syntax where it's used.
>
> The delegates are just a way of passing a block of code around. If you
> just use it in place then maybe one could do like this:
>
> @(int foo;);
> return @(lhs@(op)rhs);
>
Yea, my point was just that the "@..." stuff could work either way, with the
string-based system or with your delegate-based one.
I don't mean to come across like I'm ignoring or against the idea of the
whole delegate aspect, and I understand that the main point of the OP is to
replace the strings with delegates, but with the q{...} syntax and
string-templating, I'm still struggling to see a big enough benefit compared
to the status quo. I see that using delegates instead of strings could
probably be made to work, but my questions are "For what benefit(s)?" and
"Would those benefits be sufficient to warrant the change?" I'm not
necessarily saying the answer is "no", but I'm unconvinced so far.
And here's another thing: Suppose we got a Ruby/PHP-like syntax for
embedding code substitutions directly into a string (which would have other
useful applications besides mixins):
auto name = "Joe";
auto msg = "hello #{name}, whaddup?";
mixin( q{ int #{name} = 7; } );
Joe++;
Would that eliminate much (or all) of the benefit of the delegate approach?
Re: New syntax for string mixins
"Jacob Carlborg" wrote in message news:iednio$2vj...@digitalmars.com... > > I can't quite visualize how the final code will look like and as you say > it's hard without a formal definition of AST macros. But I think somehow > it would be possible, I mean instead of combining strings one combine > expressions/syntax. But I don't know if it would be possible to combine > incomplete expressions. > One parallel that may or may not be applicable, but might be worth considering, is dynamically building XHTML: Using a string-template system is generally found to work very well, but building it via a DOM (essentially an AST) is often considered a bit of a pain. I guess the simplest take-away would be that string-based approaches may be easier get working well. FWIW.
Re: New syntax for string mixins
On 2010-12-16 11:18, Pelle Månsson wrote:
On 12/15/2010 11:00 PM, Nick Sabalausky wrote:
"Jonathan M Davis" wrote in message
news:mailman.1035.1292441722.21107.digitalmar...@puremagic.com...
On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
That was my idea as well, that
@get_set("int", "bar");
could be translated into
mixin(get_set("int", "bar")); just like
just like scope statements are translated into try/catch/finally.
Honestly, I don't see much gain in using @ rather than mixin(). It's a
little
less typing, but that's it.
It does seem like a small difference, just replacing "mixin" with "@" and
removing one layer of parens. But I think that extra layer of parens,
minor
as it seems, makes a big difference in the readability (and
"typeability")
of mixin invocations. Those extra parens do get to be a real bother,
major
visual noise at least to my eyes.
I agree with this. Actually, just removing the parenthesis would be a
huge gain for me.
And it precludes stuff like mixin("lhs " ~ op ~ "
rhs") like happens all the time in overloaded operator functions.
I don't see why these shouldn't work:
@"int foo;";
return @("lhs " ~ op ~ " rhs");
At least with just the "@" part of the proposal. Maybe the delegate thing
might make it tricker, I dunno.
This could work, but I don't think anyone is suggesting completely
replacing the mixin. I think @ could be a function/template thing, and
have strings in a more explicit mixin"";
Then again, inconsistency sucks.
The whole point of the idea/suggestion was to get rid of the strings.
Then if the syntax I've suggested is translated into the string mixin
syntax we have now then I would be fine with that.
--
/Jacob Carlborg
Re: New syntax for string mixins
On 2010-12-15 23:00, Nick Sabalausky wrote:
"Jonathan M Davis" wrote in message
news:mailman.1035.1292441722.21107.digitalmar...@puremagic.com...
On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
That was my idea as well, that
@get_set("int", "bar");
could be translated into
mixin(get_set("int", "bar")); just like
just like scope statements are translated into try/catch/finally.
Honestly, I don't see much gain in using @ rather than mixin(). It's a
little
less typing, but that's it.
It does seem like a small difference, just replacing "mixin" with "@" and
removing one layer of parens. But I think that extra layer of parens, minor
as it seems, makes a big difference in the readability (and "typeability")
of mixin invocations. Those extra parens do get to be a real bother, major
visual noise at least to my eyes.
^^ I completely agree.
And it precludes stuff like mixin("lhs " ~ op ~ "
rhs") like happens all the time in overloaded operator functions.
I don't see why these shouldn't work:
@"int foo;";
return @("lhs " ~ op ~ " rhs");
At least with just the "@" part of the proposal. Maybe the delegate thing
might make it tricker, I dunno.
My idea was actually to get rid of the strings where the code to be
mixed in is defined and to have a better syntax where it's used.
The delegates are just a way of passing a block of code around. If you
just use it in place then maybe one could do like this:
@(int foo;);
return @(lhs@(op)rhs);
--
/Jacob Carlborg
Re: New syntax for string mixins
On 2010-12-15 08:57, Don wrote:
Jacob Carlborg wrote:
On 2010-12-14 13:05, Don wrote:
Graham St Jack wrote:
On 14/12/10 20:33, Vladimir Panteleev wrote:
On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack
wrote:
There is of course the worry that it could get so easy that everyone
starts doing it, and we have (relatively) impenetrable code
everywhere
instead of just deep in the bowels of framework libraries.
TBH, I'm more excited by AST macros which I understood are planned for
D3, as mentioned here:
http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf
They seem to promise the power of mixins, but without the mess.
I took a look at the pdf, but couldn't see how the AST macros could
come
close to the kinds of things that are possible (but difficult) with
mixins.
That fact was recognized at the conference, on the following day. As a
result, AST macros were dropped from D2.
Do you have an example that would work with string mixins but not with
AST macros?
Well, it's a bit hard without a formal definition of AST macros.
But the stuff I talked about at the conference, I have no idea how to do
with AST macros.
There's code like this, which generates an asm instruction.
--
mixin( opToSSE[operations[done+1]] ~ suffix ~ " " ~ XMM(numOnStack-1) ~
", " ~ indexedSSEVector(ranklist, operations[done], vectorsize));
---
using the functions:
const char [][5] vectorRegister = ["ECX", "EDX", "EBX", "ESI", "EDI"];
char [] indexedSSEVector(char [] ranklist, char var, char [] vecsize)
{
return "[" ~ vectorRegister[vectorNum(ranklist, var)] ~ " + " ~ vecsize
~"*EAX]";
}
char [] XMM(int k) { return "XMM"~ itoa(k); }
char [][char] opToSSE() {
return ['*':"mulp"[], '+': "addp", '-': "subp", '/': "divp"]; }
int vectorNum(char [] ranklist, char var)
{
int numVecs=0;
for (int i=0; i
I can't quite visualize how the final code will look like and as you say
it's hard without a formal definition of AST macros. But I think somehow
it would be possible, I mean instead of combining strings one combine
expressions/syntax. But I don't know if it would be possible to combine
incomplete expressions.
They need to roughly match string mixins in power. At this stage, there
is no proposal for how they should work.
I think someone, Nick Sabalausky perhaps, suggested to have something
like the hygiene macros in Nemerle:
http://nemerle.org/wiki/index.php?title=Macros
From an implementation point of view, the differences between Nemerle
macros and string mixins are mostly syntactic.
The one thing about them that I find really impressive is the IDE
integration, especially that they got syntax highlighting to work. I
don't know they've done that.
As far as I can see the content of a macro in Nemerle is just code. But
if you're referring to the syntax expression/statement which adds new
syntax to the language then I agree.
--
/Jacob Carlborg
Re: New syntax for string mixins
On 2010-12-15 00:19, Graham St Jack wrote:
I don't know, do you have an example ?
For example taking a classname and a bunch of field types and names, and
turning it into a class definition complete with constructor from field
values, constructor from an input stream, a method to write to an output
stream, and const getters. Or maybe std.typecons.AutoImplement.
Could you post an example of how that mixin would be used and the code
it would generate then I can see if I can translate it to my syntax.
AutoImplement seems to just contain template mixins which is something
else.
I have attached my concurrency framework, which relies heavily on
mixins, plus its unit test to show how it is used. I haven't included
the various dependencies because I assume you just want the example
code. Let me know if you want something buildable, or perhaps something
more cut-down.
What the code-generating template does is to create from something like
this (you can have any number of Messages in a Protocol):
alias Protocol!("Requests", Message!("job", string, "name")).code jobCode;
mixin(jobCode);
this code:
class Requests {
struct jobMsg {
string name;
this(string name) {
this.name = name;
}
void read(InStream stream) {
name = stream.get!string;
}
void write(OutStream stream) {
stream(name);
}
}
struct Message {
uint kind;
union {
jobMsg job;
}
this(ref jobMsg msg) {
kind = 0;
job = msg;
}
this(InStream stream) {
kind = stream.get!uint;
switch(kind) {
case 0: job.read(stream); break;
default: assert(0, "Cannot read unsupported message kind");
}
}
void write(OutStream stream) {
stream(kind);
switch(kind) {
case 0: job.write(stream); break;
default: assert(0, "Cannot write unsupported message kind");
}
}
}
private alias Channel!(Message) _Chan;
private alias shared _Chan Chan;
private Chan channel;
this() { channel = new Chan(); }
ChannelSelectable newSelectable() { return channel.newSelectable(); }
void finalize() { channel.finalize; }
interface IHandler {
void job(string name);
}
void job(string name) {
channel.add(Message(jobMsg(name)));
}
void receive(IHandler handler) {
auto message = channel.remove;
switch (message.kind) {
case 0: handler.job(message.job.name); break;
default: assert(0, "Cannot dispatch unsupported message kind");
}
}
}
I use this for inter-thread communications, and I use the discriminated
union to pass messages between processes. The manual mixin after the
alias is a debugging aid - the compiler errors when doing mixins aren't
helpful at all. I case you are wondering why I did all this, it was
partly a learning experience, but mostly an attempt to do something
properly thread-safe (!hasAliasing), using shared, const and immutable
properly.
I've attached a part of how concurrency.d could look like translated to
my suggested syntax. It probably contains a lot of errors because did a
quick translation and I had some trouble understanding the mixins.
--
/Jacob Carlborg
void delegate () Paste (string name, string joiner)
{
return
{
static if (more)
enum @name = { _...@name @joiner Next.name; };
else
enum @name = { _...@name; };
}
}
private template MessageStrings(string msgName, T...) {
static assert(T.length > 1, "Message parameters must be in pairs");
enum more = T.length > 2;
static if (more) {
alias MessageStrings!(msgName, T[2..$]) Next;
}
static if (T.length > 0) {
static assert(!hasAliasing!(T[0]), "Cannot use type " ~ T[0].stringof ~
" in a message");
static assert(is(typeof(T[1]) : string), "Message parameters must be
named");
enum _fieldStr = { @(T[0].stringof) @T[1]; };
enum _initStr = { th...@t[1] = @T[1]; }
enum _readStr = { @t[1] = stream.g...@t[0].stringof; }
enum _writeStr = { (@T[1]) };
enum _paramStr = { @(T[0].stringof) @T[1]; };
enum _nameStr = {�...@t[1]; };
enum _callStr = { messa...@msgname.@T[1]; };
@Paste("fieldStr", "\n"); // field types and names in
definition
@Paste("initStr", "\n"); // field assignments in
constructor
@Paste("readStr", "\n"); // fields read from an
InStream
@Paste("writeStr", ""); // fields written to an
OutStream
@Paste("paramStr", ", "); // field types and names,
comma-separated
@Paste("nameStr", ", "); // field names, comma
separated
@Paste("callStr", ", "); // field values accessed in
an enclosing union, comma separated
}
}
void delegate () Messages (T...) (string name)
{
static assert(T.length > 0, "Messages have to contain fields");
alias name msgName;
alias MessageStrings!(name, T) strings;
return
{
Re: New syntax for string mixins
On 12/15/2010 11:00 PM, Nick Sabalausky wrote:
"Jonathan M Davis" wrote in message
news:mailman.1035.1292441722.21107.digitalmar...@puremagic.com...
On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
That was my idea as well, that
@get_set("int", "bar");
could be translated into
mixin(get_set("int", "bar")); just like
just like scope statements are translated into try/catch/finally.
Honestly, I don't see much gain in using @ rather than mixin(). It's a
little
less typing, but that's it.
It does seem like a small difference, just replacing "mixin" with "@" and
removing one layer of parens. But I think that extra layer of parens, minor
as it seems, makes a big difference in the readability (and "typeability")
of mixin invocations. Those extra parens do get to be a real bother, major
visual noise at least to my eyes.
I agree with this. Actually, just removing the parenthesis would be a
huge gain for me.
And it precludes stuff like mixin("lhs " ~ op ~ "
rhs") like happens all the time in overloaded operator functions.
I don't see why these shouldn't work:
@"int foo;";
return @("lhs " ~ op ~ " rhs");
At least with just the "@" part of the proposal. Maybe the delegate thing
might make it tricker, I dunno.
This could work, but I don't think anyone is suggesting completely
replacing the mixin. I think @ could be a function/template thing, and
have strings in a more explicit mixin"";
Then again, inconsistency sucks.
Re: New syntax for string mixins
"Jonathan M Davis" wrote in message
news:mailman.1035.1292441722.21107.digitalmar...@puremagic.com...
> On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
>>
>> That was my idea as well, that
>>
>> @get_set("int", "bar");
>>
>> could be translated into
>>
>> mixin(get_set("int", "bar")); just like
>>
>> just like scope statements are translated into try/catch/finally.
>
> Honestly, I don't see much gain in using @ rather than mixin(). It's a
> little
> less typing, but that's it.
It does seem like a small difference, just replacing "mixin" with "@" and
removing one layer of parens. But I think that extra layer of parens, minor
as it seems, makes a big difference in the readability (and "typeability")
of mixin invocations. Those extra parens do get to be a real bother, major
visual noise at least to my eyes.
> And it precludes stuff like mixin("lhs " ~ op ~ "
> rhs") like happens all the time in overloaded operator functions.
>
I don't see why these shouldn't work:
@"int foo;";
return @("lhs " ~ op ~ " rhs");
At least with just the "@" part of the proposal. Maybe the delegate thing
might make it tricker, I dunno.
Re: New syntax for string mixins
On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
> On 2010-12-14 21:44, Nick Sabalausky wrote:
> > "Jacob Carlborg" wrote in message
> > news:ie8i8c$15f...@digitalmars.com...
> >
> >> On 2010-12-14 19:13, Nick Sabalausky wrote:
> >>> "Graham St Jack" wrote in message
> >>> news:ie76ig$b2...@digitalmars.com...
> >>>
> What you are suggesting here seems to be a way to dramatically reduce
> the
> complexity of code that generates source-code and mixes it in. I think
> something like that is needed before this mind-bogglingly powerful
> feature
> of D can realise its potential.
> >>>
> >>> I think a decent string-template library could probably come very close
> >>> to
> >>> the proposal without needing any language changes at all:
> >>>
> >>> string get_set(T, string name)()
> >>> {
> >>>
> >>>return
> >>>q{
> >>>
> >>>@type _...@name;
> >>>
> >>>@type @name ()
> >>>{
> >>>
> >>>return _...@name;
> >>>
> >>>}
> >>>
> >>>@type @name (@type @name)
> >>>{
> >>>
> >>>return _...@name = @name;
> >>>
> >>>}
> >>>
> >>>}.replace( ["@type": T.stringof, "@name": name] );
> >>>
> >>> }
> >>>
> >>> class Foo
> >>> {
> >>>
> >>>mixin(get_set!(int, "bar")());
> >>>
> >>> }
> >>>
> >>> There are definitely some things about the proposal that are better
> >>> than with this, but I just wanted to point out that the value of the
> >>> proposal should probably be evaluated against something roughly like
> >>> the above rather
> >>> than something that does a bunch of procedural string splicing.
> >>
> >> The whole point of the idea was to get rid of the strings and the mixin
> >> expression (as it looks like to day).
> >
> > While I'm not necessarily opposed to the idea of getting rid of the
> > strings, I guess I don't really see what improvement your proposal
> > provides other than not having to manually specify the mapping of
> > "replace what identifier with what data".
> >
> > Getting rid of the "mixin" I see as a separate issue. We could just as
> >
> > easily say that given the function "string get_set(...)":
> > @get_set("int", "bar");
> > or
> > @get_set int bar;
> >
> > ...Is shorthand for, or is even the new syntax for:
> > mixin(get_set("int", "bar"));
>
> That was my idea as well, that
>
> @get_set("int", "bar");
>
> could be translated into
>
> mixin(get_set("int", "bar")); just like
>
> just like scope statements are translated into try/catch/finally.
Honestly, I don't see much gain in using @ rather than mixin(). It's a little
less typing, but that's it. And it precludes stuff like mixin("lhs " ~ op ~ "
rhs") like happens all the time in overloaded operator functions.
- Jonathan M Davis
Re: New syntax for string mixins
On 2010-12-14 21:44, Nick Sabalausky wrote:
"Jacob Carlborg" wrote in message
news:ie8i8c$15f...@digitalmars.com...
On 2010-12-14 19:13, Nick Sabalausky wrote:
"Graham St Jack" wrote in message
news:ie76ig$b2...@digitalmars.com...
What you are suggesting here seems to be a way to dramatically reduce
the
complexity of code that generates source-code and mixes it in. I think
something like that is needed before this mind-bogglingly powerful
feature
of D can realise its potential.
I think a decent string-template library could probably come very close
to
the proposal without needing any language changes at all:
string get_set(T, string name)()
{
return
q{
@type _...@name;
@type @name ()
{
return _...@name;
}
@type @name (@type @name)
{
return _...@name = @name;
}
}.replace( ["@type": T.stringof, "@name": name] );
}
class Foo
{
mixin(get_set!(int, "bar")());
}
There are definitely some things about the proposal that are better than
with this, but I just wanted to point out that the value of the proposal
should probably be evaluated against something roughly like the above
rather
than something that does a bunch of procedural string splicing.
The whole point of the idea was to get rid of the strings and the mixin
expression (as it looks like to day).
While I'm not necessarily opposed to the idea of getting rid of the strings,
I guess I don't really see what improvement your proposal provides other
than not having to manually specify the mapping of "replace what identifier
with what data".
Getting rid of the "mixin" I see as a separate issue. We could just as
easily say that given the function "string get_set(...)":
@get_set("int", "bar");
or
@get_set int bar;
...Is shorthand for, or is even the new syntax for:
mixin(get_set("int", "bar"));
That was my idea as well, that
@get_set("int", "bar");
could be translated into
mixin(get_set("int", "bar")); just like
just like scope statements are translated into try/catch/finally.
--
/Jacob Carlborg
Re: New syntax for string mixins
Jacob Carlborg wrote:
On 2010-12-14 13:05, Don wrote:
Graham St Jack wrote:
On 14/12/10 20:33, Vladimir Panteleev wrote:
On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack
wrote:
There is of course the worry that it could get so easy that everyone
starts doing it, and we have (relatively) impenetrable code everywhere
instead of just deep in the bowels of framework libraries.
TBH, I'm more excited by AST macros which I understood are planned for
D3, as mentioned here:
http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf
They seem to promise the power of mixins, but without the mess.
I took a look at the pdf, but couldn't see how the AST macros could come
close to the kinds of things that are possible (but difficult) with
mixins.
That fact was recognized at the conference, on the following day. As a
result, AST macros were dropped from D2.
Do you have an example that would work with string mixins but not with
AST macros?
Well, it's a bit hard without a formal definition of AST macros.
But the stuff I talked about at the conference, I have no idea how to do
with AST macros.
There's code like this, which generates an asm instruction.
--
mixin( opToSSE[operations[done+1]] ~ suffix ~ " " ~ XMM(numOnStack-1) ~
", " ~ indexedSSEVector(ranklist, operations[done], vectorsize));
---
using the functions:
const char [][5] vectorRegister = ["ECX", "EDX", "EBX", "ESI", "EDI"];
char [] indexedSSEVector(char [] ranklist, char var, char [] vecsize)
{
return "[" ~ vectorRegister[vectorNum(ranklist, var)] ~ " + " ~
vecsize ~"*EAX]";
}
char [] XMM(int k) { return "XMM"~ itoa(k); }
char [][char] opToSSE() {
return ['*':"mulp"[], '+': "addp", '-': "subp", '/': "divp"]; }
int vectorNum(char [] ranklist, char var)
{
int numVecs=0;
for (int i=0; i
They need to roughly match string mixins in power. At this stage, there
is no proposal for how they should work.
I think someone, Nick Sabalausky perhaps, suggested to have something
like the hygiene macros in Nemerle:
http://nemerle.org/wiki/index.php?title=Macros
From an implementation point of view, the differences between Nemerle
macros and string mixins are mostly syntactic.
The one thing about them that I find really impressive is the IDE
integration, especially that they got syntax highlighting to work. I
don't know they've done that.
Re: New syntax for string mixins
I don't know, do you have an example ?
For example taking a classname and a bunch of field types and names, and
turning it into a class definition complete with constructor from field
values, constructor from an input stream, a method to write to an output
stream, and const getters. Or maybe std.typecons.AutoImplement.
Could you post an example of how that mixin would be used and the code
it would generate then I can see if I can translate it to my syntax.
AutoImplement seems to just contain template mixins which is something
else.
I have attached my concurrency framework, which relies heavily on
mixins, plus its unit test to show how it is used. I haven't included
the various dependencies because I assume you just want the example
code. Let me know if you want something buildable, or perhaps something
more cut-down.
What the code-generating template does is to create from something like
this (you can have any number of Messages in a Protocol):
alias Protocol!("Requests",Message!("job",string, "name")).code
jobCode;
mixin(jobCode);
this code:
class Requests {
struct jobMsg {
string name;
this(string name) {
this.name = name;
}
void read(InStream stream) {
name = stream.get!string;
}
void write(OutStream stream) {
stream(name);
}
}
struct Message {
uint kind;
union {
jobMsg job;
}
this(ref jobMsg msg) {
kind = 0;
job = msg;
}
this(InStream stream) {
kind = stream.get!uint;
switch(kind) {
case 0: job.read(stream); break;
default: assert(0, "Cannot read unsupported message kind");
}
}
void write(OutStream stream) {
stream(kind);
switch(kind) {
case 0: job.write(stream); break;
default: assert(0, "Cannot write unsupported message
kind");
}
}
}
private alias Channel!(Message) _Chan;
private alias shared _Chan Chan;
private Chan channel;
this() { channel = new Chan(); }
ChannelSelectable newSelectable() { return channel.newSelectable(); }
void finalize() { channel.finalize; }
interface IHandler {
void job(string name);
}
void job(string name) {
channel.add(Message(jobMsg(name)));
}
void receive(IHandler handler) {
auto message = channel.remove;
switch (message.kind) {
case 0: handler.job(message.job.name); break;
default: assert(0, "Cannot dispatch unsupported message kind");
}
}
}
I use this for inter-thread communications, and I use the discriminated
union to pass messages between processes. The manual mixin after the
alias is a debugging aid - the compiler errors when doing mixins aren't
helpful at all. I case you are wondering why I did all this, it was
partly a learning experience, but mostly an attempt to do something
properly thread-safe (!hasAliasing), using shared, const and immutable
properly.
--
Graham St Jack
module bedrock.maxim.concurrency;
public import bedrock.maxim.io;
public import bedrock.maxim.stream;
import bedrock.maxim.logging;
import std.algorithm;
import std.stdio;
import std.conv;
import std.traits;
import std.typecons;
import std.typetuple;
import core.thread;
import core.stdc.stdlib;
import core.stdc.errno;
import core.stdc.config;
import core.sys.posix.pthread;
import core.sys.posix.signal;
static import linux = std.c.linux.linux;
//
// Provides a message-passing concurrency implementation.
// * Messages cannot contain references to mutable data.
// * Message queues (Channels) are explicitly created and passed to threads as
// arguments to spawn().
// * Channels are synchronized (and thus shared).
// * The types of messages a Channel supports are well-defined.
// * Messages are processed in order.
// * Multiple threads can receive from the same Channel.
// * Channels have an internal pipe that can be used in a Selector to allow
// a thread to receive input from multiple Channels and file-descriptor devices
// such as sockets and pipes.
// * There is no automatic thread-termination behaviour.
//
// TODO - prevent non-shared aliased data from being passed to spawn().
// TODO - gathering non-fatal signals via signalfd, putting info on a Channel.
//
//-
// Condition - Adapted from core.sync.condition to be linux-specific,
// use an object's monitor and to be shared.
//-
public class ConditionException : Exception {
this(string text) { super(text); }
}
// clock_gettime definitions copied from commented-out and scattered
// definitions in druntime
private {
enum int CLOCK_REALTIME = 0;
enum int TIMER_ABSTIME = 0x01
Re: New syntax for string mixins
"Jacob Carlborg" wrote in message
news:ie8i8c$15f...@digitalmars.com...
> On 2010-12-14 19:13, Nick Sabalausky wrote:
>> "Graham St Jack" wrote in message
>> news:ie76ig$b2...@digitalmars.com...
>>>
>>> What you are suggesting here seems to be a way to dramatically reduce
>>> the
>>> complexity of code that generates source-code and mixes it in. I think
>>> something like that is needed before this mind-bogglingly powerful
>>> feature
>>> of D can realise its potential.
>>>
>>
>> I think a decent string-template library could probably come very close
>> to
>> the proposal without needing any language changes at all:
>>
>> string get_set(T, string name)()
>> {
>> return
>> q{
>> @type _...@name;
>>
>> @type @name ()
>> {
>> return _...@name;
>> }
>>
>> @type @name (@type @name)
>> {
>> return _...@name = @name;
>> }
>> }.replace( ["@type": T.stringof, "@name": name] );
>> }
>>
>> class Foo
>> {
>> mixin(get_set!(int, "bar")());
>> }
>>
>> There are definitely some things about the proposal that are better than
>> with this, but I just wanted to point out that the value of the proposal
>> should probably be evaluated against something roughly like the above
>> rather
>> than something that does a bunch of procedural string splicing.
>
> The whole point of the idea was to get rid of the strings and the mixin
> expression (as it looks like to day).
>
While I'm not necessarily opposed to the idea of getting rid of the strings,
I guess I don't really see what improvement your proposal provides other
than not having to manually specify the mapping of "replace what identifier
with what data".
Getting rid of the "mixin" I see as a separate issue. We could just as
easily say that given the function "string get_set(...)":
@get_set("int", "bar");
or
@get_set int bar;
...Is shorthand for, or is even the new syntax for:
mixin(get_set("int", "bar"));
Re: New syntax for string mixins
On 2010-12-14 19:13, Nick Sabalausky wrote:
"Graham St Jack" wrote in message
news:ie76ig$b2...@digitalmars.com...
What you are suggesting here seems to be a way to dramatically reduce the
complexity of code that generates source-code and mixes it in. I think
something like that is needed before this mind-bogglingly powerful feature
of D can realise its potential.
I think a decent string-template library could probably come very close to
the proposal without needing any language changes at all:
string get_set(T, string name)()
{
return
q{
@type _...@name;
@type @name ()
{
return _...@name;
}
@type @name (@type @name)
{
return _...@name = @name;
}
}.replace( ["@type": T.stringof, "@name": name] );
}
class Foo
{
mixin(get_set!(int, "bar")());
}
There are definitely some things about the proposal that are better than
with this, but I just wanted to point out that the value of the proposal
should probably be evaluated against something roughly like the above rather
than something that does a bunch of procedural string splicing.
The whole point of the idea was to get rid of the strings and the mixin
expression (as it looks like to day).
--
/Jacob Carlborg
Re: New syntax for string mixins
"Jacob Carlborg" wrote in message news:ie8dpq$kf...@digitalmars.com... > On 2010-12-14 13:05, Don wrote: >> Graham St Jack wrote: >>> On 14/12/10 20:33, Vladimir Panteleev wrote: On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack wrote: > There is of course the worry that it could get so easy that everyone > starts doing it, and we have (relatively) impenetrable code everywhere > instead of just deep in the bowels of framework libraries. TBH, I'm more excited by AST macros which I understood are planned for D3, as mentioned here: http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf They seem to promise the power of mixins, but without the mess. >>> >>> I took a look at the pdf, but couldn't see how the AST macros could come >>> close to the kinds of things that are possible (but difficult) with >>> mixins. >> >> That fact was recognized at the conference, on the following day. As a >> result, AST macros were dropped from D2. > > Do you have an example that would work with string mixins but not with AST > macros? > >> They need to roughly match string mixins in power. At this stage, there >> is no proposal for how they should work. > > I think someone, Nick Sabalausky perhaps, suggested to have something like > the hygiene macros in Nemerle: > http://nemerle.org/wiki/index.php?title=Macros > Though I'm a huge D/systems-language/native-compiled guy, Nemerle's macros and pattern matching are two things I'm very jealous of and have made it very tempting to put up with the CLR for certain things.
Re: New syntax for string mixins
On 2010-12-14 13:05, Don wrote: Graham St Jack wrote: On 14/12/10 20:33, Vladimir Panteleev wrote: On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack wrote: There is of course the worry that it could get so easy that everyone starts doing it, and we have (relatively) impenetrable code everywhere instead of just deep in the bowels of framework libraries. TBH, I'm more excited by AST macros which I understood are planned for D3, as mentioned here: http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf They seem to promise the power of mixins, but without the mess. I took a look at the pdf, but couldn't see how the AST macros could come close to the kinds of things that are possible (but difficult) with mixins. That fact was recognized at the conference, on the following day. As a result, AST macros were dropped from D2. Do you have an example that would work with string mixins but not with AST macros? They need to roughly match string mixins in power. At this stage, there is no proposal for how they should work. I think someone, Nick Sabalausky perhaps, suggested to have something like the hygiene macros in Nemerle: http://nemerle.org/wiki/index.php?title=Macros -- /Jacob Carlborg
Re: New syntax for string mixins
On 2010-12-14 12:42, Graham St Jack wrote: On 14/12/10 20:33, Vladimir Panteleev wrote: On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack wrote: There is of course the worry that it could get so easy that everyone starts doing it, and we have (relatively) impenetrable code everywhere instead of just deep in the bowels of framework libraries. TBH, I'm more excited by AST macros which I understood are planned for D3, as mentioned here: http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf They seem to promise the power of mixins, but without the mess. I took a look at the pdf, but couldn't see how the AST macros could come close to the kinds of things that are possible (but difficult) with mixins. Is there more information somewhere? Jacob, I can see how your proposed syntax would make simple mixins easier and clearer, but how would it do something more complex? I don't know, do you have an example ? For example taking a classname and a bunch of field types and names, and turning it into a class definition complete with constructor from field values, constructor from an input stream, a method to write to an output stream, and const getters. Or maybe std.typecons.AutoImplement. Could you post an example of how that mixin would be used and the code it would generate then I can see if I can translate it to my syntax. AutoImplement seems to just contain template mixins which is something else. -- /Jacob Carlborg
Re: New syntax for string mixins
"Graham St Jack" wrote in message
news:ie76ig$b2...@digitalmars.com...
>
> What you are suggesting here seems to be a way to dramatically reduce the
> complexity of code that generates source-code and mixes it in. I think
> something like that is needed before this mind-bogglingly powerful feature
> of D can realise its potential.
>
I think a decent string-template library could probably come very close to
the proposal without needing any language changes at all:
string get_set(T, string name)()
{
return
q{
@type _...@name;
@type @name ()
{
return _...@name;
}
@type @name (@type @name)
{
return _...@name = @name;
}
}.replace( ["@type": T.stringof, "@name": name] );
}
class Foo
{
mixin(get_set!(int, "bar")());
}
There are definitely some things about the proposal that are better than
with this, but I just wanted to point out that the value of the proposal
should probably be evaluated against something roughly like the above rather
than something that does a bunch of procedural string splicing.
Re: New syntax for string mixins
On 2010-12-14 11:03, Vladimir Panteleev wrote: On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack wrote: There is of course the worry that it could get so easy that everyone starts doing it, and we have (relatively) impenetrable code everywhere instead of just deep in the bowels of framework libraries. TBH, I'm more excited by AST macros which I understood are planned for D3, as mentioned here: http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf They seem to promise the power of mixins, but without the mess. I would like to have AST macros too but I was think my suggestion was perhaps easier to implement, just some syntactic sugar. -- /Jacob Carlborg
Re: New syntax for string mixins
Graham St Jack wrote: On 14/12/10 20:33, Vladimir Panteleev wrote: On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack wrote: There is of course the worry that it could get so easy that everyone starts doing it, and we have (relatively) impenetrable code everywhere instead of just deep in the bowels of framework libraries. TBH, I'm more excited by AST macros which I understood are planned for D3, as mentioned here: http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf They seem to promise the power of mixins, but without the mess. I took a look at the pdf, but couldn't see how the AST macros could come close to the kinds of things that are possible (but difficult) with mixins. That fact was recognized at the conference, on the following day. As a result, AST macros were dropped from D2. They need to roughly match string mixins in power. At this stage, there is no proposal for how they should work.
Re: New syntax for string mixins
On 14/12/10 20:33, Vladimir Panteleev wrote: On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack wrote: There is of course the worry that it could get so easy that everyone starts doing it, and we have (relatively) impenetrable code everywhere instead of just deep in the bowels of framework libraries. TBH, I'm more excited by AST macros which I understood are planned for D3, as mentioned here: http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf They seem to promise the power of mixins, but without the mess. I took a look at the pdf, but couldn't see how the AST macros could come close to the kinds of things that are possible (but difficult) with mixins. Is there more information somewhere? Jacob, I can see how your proposed syntax would make simple mixins easier and clearer, but how would it do something more complex? For example taking a classname and a bunch of field types and names, and turning it into a class definition complete with constructor from field values, constructor from an input stream, a method to write to an output stream, and const getters. Or maybe std.typecons.AutoImplement. -- Graham St Jack
Re: New syntax for string mixins
On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack wrote: There is of course the worry that it could get so easy that everyone starts doing it, and we have (relatively) impenetrable code everywhere instead of just deep in the bowels of framework libraries. TBH, I'm more excited by AST macros which I understood are planned for D3, as mentioned here: http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf They seem to promise the power of mixins, but without the mess. -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: New syntax for string mixins
I have done a fair bit of mixin coding using recursive templates
(inspired by std.typecons). It was an amazing taste of what you can do
in D, and I am delighted with the result - HEAPS of boiler-plate coding
vanished before my eyes. However, the template code is virtually
impossible to understand.
What you are suggesting here seems to be a way to dramatically reduce
the complexity of code that generates source-code and mixes it in. I
think something like that is needed before this mind-bogglingly powerful
feature of D can realise its potential.
There is of course the worry that it could get so easy that everyone
starts doing it, and we have (relatively) impenetrable code everywhere
instead of just deep in the bowels of framework libraries.
On 14/12/10 07:07, Jacob Carlborg wrote:
This is an idea I've been thinking of for a while, it's not a really
suggestion (at least not yet) I just wanted to here what people think
about it.
If we take a step back and look at what string mixins actually do or
rather what they're used for, that would be: inserting a piece/block
of code where the mixin expression is used. If we then take a look at
how a block of code is represented in D (how you store it in variables
and how you pass it around). It's not as a string which is used by the
mixin expression, instead delegates are used to represent a block of
code in D that can be passed around. Therefore this is my idea:
Add a new mixin operator "@" (1). When that operator is put in front
of a function call it would behave as a string mixin. The function
that is called needs to be CTFE and return a delegate or an array of
delegates:
class Foo
{
@get_set("int", "bar");
}
The above code would be the same as the following code:
class Foo
{
mixin(get_set("int", "bar"));
}
If we now move to the declaration of "get_set" this is how it could
look like:
void delegate () get_set (string type, string name)
{
return
{
@type _...@name;
@type @name ()
{
return _...@name;
}
@type @name (@type @name)
{
return _...@name = @name;
}
};
}
In the above code when "@" is used in the delegate literal it
basically behaves like string interpolation, so "@type" would be
replaced with the content of the "type" variable (2).
When the "get_set" function is called with the mixin symbol the
content of the returned delegate is inserted where the call is made.
If the function returns an array of delegates then the array would be
unfolded and the content of all the delegates would be inserted.
Taking it one step further:
Allow the mixin syntax to be placed in front of most of the
declarations and drop the need for string literals, commas and
parentheses, basically allowing the following syntax:
class Foo
{
@get_set int bar;
}
Would be translated into this:
class Foo
{
@get_set!(int)("bar");
}
Maybe one could do something like this as well:
@singleton class Foo
{
}
void delegate () singleton (string name, void delegate () classBody)
{
return { // a simple singleton implementation
class @name
{
static @name instance;
static this ()
{
instance = new @name;
}
private this () {}
@classBody;
}
};
}
Above, in the last example, @classBody would insert the content of the
delegate, basically the class body.
1. I will use "@" in this example because I think it looks good but it
would probably conflict with @nothrow, @property and others.
2. I have no idea if this usage of the mixin symbol, "@", will
conflict with the first usage.
--
Graham St Jack
