Re: [proto] [phoenix3] New design proposal
On Wednesday 20 October 2010 05:19:17 Joel de Guzman wrote: > On 10/20/2010 12:08 AM, Eric Niebler wrote: > > On 10/19/2010 1:33 AM, Thomas Heller wrote: > >> On Tue, Oct 19, 2010 at 6:21 AM, Joel de Guzman wrote: > >>> Can we also focus on one very specific use-case that demonstrates > >>> the motivation behind the need of such a refactoring and why the > >>> old(er) design is not sufficient? I'd really want to sync up with > >>> you guys. > >> > >> With the old design (the one which is currently in the gsoc svn > >> sandbox) I had problems with defining what phoenix expressions really > >> are. We had at least two types of expressions. First were the ones we > >> reused from proto (plus, multiplies, function and so on), Second were > >> these proto::function constructs which had a funcwrap struct and > >> an env placeholder. This env placeholder just wastes a valuable slot > >> for potential arguments. The second point why this design is not > >> good, is that data and behaviour is not separated. The T in funcwrap > >> defines how the phoenix expression will get evaluated. > >> > >> This design solves this two problems: Data and behaviour are cleanly > >> separated. Additionally we end up with only one type of expressions: > >> A expression is a structure which has a tag, and a variable list of > >> children. You define what what a valid expression is by extending the > >> phoenix_algorithm template through specialisation for your tag. The > >> Actions parameter is responsible for evaluating the expression. By > >> template parametrisation of this parameter we allow users to easily > >> define their own evaluation schemes without worrying about the > >> validity of the phoenix expression. This is fixed by the meta grammar > >> class. > > > > What Thomas said. We realized that for Phoenix to be extensible at the > > lowest level, we'd need to document its intermediate form: the Proto > > tree. That way folks have the option to use Proto transforms on it. > > (There are higher-level customization points that don't expose Proto, > > but I'm talking about real gear-heads here.) > > > > There were ugly things about the intermediate form we wanted to clean > > up before we document it. That started the discussion. Then the > > discussion turned to, "Can a user just change a semantic actions here > > and there without having to redefine the whole Phoenix grammar in > > Proto, which is totally non-trivial?" I forget offhand what the use > > case was, but it seemed a reasonable thing to want to do in general. > > So as Thomas says, the goal is two-fold: (a) a clean-up of the > > intermediate form ahead of its documentation, and (b) a way to easily > > plug in user-defined semantic actions without changing the grammar. > > > > I think these changes effect the way to define new Phoenix syntactic > > constructs, so it's worth doing a before-and-after comparison of the > > extensibility mechanisms. Thomas, can you send around such a > > comparison? How hard is it to add a new statement, for instance? > > Yes, exactly, that's what I want. Anyway, while I'd still want to see > this, I looked at the code and I like it, except for some nits here > and there (especially naming). More on that later. Yep, naming was one of the major problems i had ;) I don't have much time now. But here is a side by side comparision of the old and new design: http://www.scribd.com/doc/39713335/Design-ion When looking at it, please keep in mind, that with the new design, we can reuse for_expr in a proto grammar. Additionally, we can reuse it in any extendable Actions class. While in the old design, everything is wrapped behind a proto::function, >. Reusing it, for example for generating debug output, is not directly supported by the old design. While in the new design, we have a unified, documented way on how to interact with out phoenix expression AST. Additionally, keep in mind, that in the old design, everything basically was a phoenix expression. The grammar wasn't really definining what can be used and what not. While in the new design, we explicitly say: here is the expression, this is how we want to use it. Sorry, I am in kind of a hurry right now. Hope that helped this far. Thomas ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] [phoenix3] New design proposal
On 10/20/2010 12:08 AM, Eric Niebler wrote: On 10/19/2010 1:33 AM, Thomas Heller wrote: On Tue, Oct 19, 2010 at 6:21 AM, Joel de Guzman wrote: Can we also focus on one very specific use-case that demonstrates the motivation behind the need of such a refactoring and why the old(er) design is not sufficient? I'd really want to sync up with you guys. With the old design (the one which is currently in the gsoc svn sandbox) I had problems with defining what phoenix expressions really are. We had at least two types of expressions. First were the ones we reused from proto (plus, multiplies, function and so on), Second were these proto::function constructs which had a funcwrap struct and an env placeholder. This env placeholder just wastes a valuable slot for potential arguments. The second point why this design is not good, is that data and behaviour is not separated. The T in funcwrap defines how the phoenix expression will get evaluated. This design solves this two problems: Data and behaviour are cleanly separated. Additionally we end up with only one type of expressions: A expression is a structure which has a tag, and a variable list of children. You define what what a valid expression is by extending the phoenix_algorithm template through specialisation for your tag. The Actions parameter is responsible for evaluating the expression. By template parametrisation of this parameter we allow users to easily define their own evaluation schemes without worrying about the validity of the phoenix expression. This is fixed by the meta grammar class. What Thomas said. We realized that for Phoenix to be extensible at the lowest level, we'd need to document its intermediate form: the Proto tree. That way folks have the option to use Proto transforms on it. (There are higher-level customization points that don't expose Proto, but I'm talking about real gear-heads here.) There were ugly things about the intermediate form we wanted to clean up before we document it. That started the discussion. Then the discussion turned to, "Can a user just change a semantic actions here and there without having to redefine the whole Phoenix grammar in Proto, which is totally non-trivial?" I forget offhand what the use case was, but it seemed a reasonable thing to want to do in general. So as Thomas says, the goal is two-fold: (a) a clean-up of the intermediate form ahead of its documentation, and (b) a way to easily plug in user-defined semantic actions without changing the grammar. I think these changes effect the way to define new Phoenix syntactic constructs, so it's worth doing a before-and-after comparison of the extensibility mechanisms. Thomas, can you send around such a comparison? How hard is it to add a new statement, for instance? Yes, exactly, that's what I want. Anyway, while I'd still want to see this, I looked at the code and I like it, except for some nits here and there (especially naming). More on that later. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] [phoenix3] New design proposal
On 10/19/2010 1:33 AM, Thomas Heller wrote: > On Tue, Oct 19, 2010 at 6:21 AM, Joel de Guzman wrote: >> Can we also focus on one very specific use-case that demonstrates >> the motivation behind the need of such a refactoring and why the >> old(er) design is not sufficient? I'd really want to sync up with >> you guys. > > With the old design (the one which is currently in the gsoc svn > sandbox) I had problems with defining what phoenix expressions really > are. We had at least two types of expressions. First were the ones we > reused from proto (plus, multiplies, function and so on), Second were > these proto::function constructs which had a funcwrap struct and > an env placeholder. This env placeholder just wastes a valuable slot > for potential arguments. The second point why this design is not > good, is that data and behaviour is not separated. The T in funcwrap > defines how the phoenix expression will get evaluated. > > This design solves this two problems: Data and behaviour are cleanly > separated. Additionally we end up with only one type of expressions: > A expression is a structure which has a tag, and a variable list of > children. You define what what a valid expression is by extending the > phoenix_algorithm template through specialisation for your tag. The > Actions parameter is responsible for evaluating the expression. By > template parametrisation of this parameter we allow users to easily > define their own evaluation schemes without worrying about the > validity of the phoenix expression. This is fixed by the meta grammar > class. What Thomas said. We realized that for Phoenix to be extensible at the lowest level, we'd need to document its intermediate form: the Proto tree. That way folks have the option to use Proto transforms on it. (There are higher-level customization points that don't expose Proto, but I'm talking about real gear-heads here.) There were ugly things about the intermediate form we wanted to clean up before we document it. That started the discussion. Then the discussion turned to, "Can a user just change a semantic actions here and there without having to redefine the whole Phoenix grammar in Proto, which is totally non-trivial?" I forget offhand what the use case was, but it seemed a reasonable thing to want to do in general. So as Thomas says, the goal is two-fold: (a) a clean-up of the intermediate form ahead of its documentation, and (b) a way to easily plug in user-defined semantic actions without changing the grammar. I think these changes effect the way to define new Phoenix syntactic constructs, so it's worth doing a before-and-after comparison of the extensibility mechanisms. Thomas, can you send around such a comparison? How hard is it to add a new statement, for instance? -- Eric Niebler BoostPro Computing http://www.boostpro.com ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] [phoenix3] New design proposal
I can imagine a lot of usecases that benefit from this feature. Let me list a few here: - Multi Stage programming: evaluate the phoenix expression to another language that can be compiled by some external compiler. The prime example i imagine for this is that someone picks that topic up, and writes a shader DSL based on phoenix reusing the already existing phoenix constructs. We already did here using a souped up phoenix like construct. SO being able to use the real thing(tm) will be even beter - Optimiziers: With the help of this Actions parameter, it almost gets trivial to write optimization passes that work on phoenix expression. I think this is worth exploring, because a optimizer working on these high level expression has way more information than for example the GIMPLE representation of GCC. see nt2 SIMD pack optimization for such use case too. ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] [phoenix3] New design proposal
On Tue, Oct 19, 2010 at 6:21 AM, Joel de Guzman wrote: > On 10/19/2010 3:25 AM, Thomas Heller wrote: >> >> Hi all, >> >> Based on recent discussions [1] I want to propose a new design for the >> intermediate structure of phoenix3. >> >> This email will be the attempt to both explain the design, and formalize >> what phoenix is, and what phoenix expressions are. >> An implementation can be found at [2]. Please feel free to comment on the >> concepts and design decision that were made. I urge you to read through >> [1] >> to better understand the motivation behind the need of such a refactoring. > > Can we also focus on one very specific use-case that demonstrates the > motivation behind the need of such a refactoring and why the old(er) > design is not sufficient? I'd really want to sync up with you guys. With the old design (the one which is currently in the gsoc svn sandbox) I had problems with defining what phoenix expressions really are. We had at least two types of expressions. First were the ones we reused from proto (plus, multiplies, function and so on), Second were these proto::function constructs which had a funcwrap struct and an env placeholder. This env placeholder just wastes a valuable slot for potential arguments. The second point why this design is not good, is that data and behaviour is not separated. The T in funcwrap defines how the phoenix expression will get evaluated. This design solves this two problems: Data and behaviour are cleanly separated. Additionally we end up with only one type of expressions: A expression is a structure which has a tag, and a variable list of children. You define what what a valid expression is by extending the phoenix_algorithm template through specialisation for your tag. The Actions parameter is responsible for evaluating the expression. By template parametrisation of this parameter we allow users to easily define their own evaluation schemes without worrying about the validity of the phoenix expression. This is fixed by the meta grammar class. I can imagine a lot of usecases that benefit from this feature. Let me list a few here: - Multi Stage programming: evaluate the phoenix expression to another language that can be compiled by some external compiler. The prime example i imagine for this is that someone picks that topic up, and writes a shader DSL based on phoenix reusing the already existing phoenix constructs. - Debugging: In order to debug a phoenix expression, we certainly do not want to evaluate the phoenix expression in a C++ context, but probably into some kind of string, giving detailed information about certain intrinsics of that expression - Optimiziers: With the help of this Actions parameter, it almost gets trivial to write optimization passes that work on phoenix expression. I think this is worth exploring, because a optimizer working on these high level expression has way more information than for example the GIMPLE representation of GCC. HTH, Thomas ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
