Re: [proto] Streamulus v0.1 is out: An EDSL for Event Stream Processing with C++
On 6/24/2012 4:42 PM, Dave Abrahams wrote: > > on Sun Jun 24 2012, Eric Niebler > wrote: > >> On 6/24/2012 8:50 AM, Irit Katriel wrote: >>> >>> In the accumulators library, all the accumulators are invoked for >>> every update to the input. This is why the visitation order can be >>> determined at compile time. >> >> That's correct. > > Are you forgetting about "droppable" accumulators? Not forgetting. It doesn't change the fact that the visitation order is set at compile time. There is no centralized, automatic, dynamic flow control in the accumulators library. -- Eric Niebler BoostPro Computing http://www.boostpro.com signature.asc Description: OpenPGP digital signature ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] Streamulus v0.1 is out: An EDSL for Event Stream Processing with C++
on Sun Jun 24 2012, Eric Niebler wrote: > On 6/24/2012 8:50 AM, Irit Katriel wrote: >> >> In the accumulators library, all the accumulators are invoked for >> every update to the input. This is why the visitation order can be >> determined at compile time. > > That's correct. Are you forgetting about "droppable" accumulators? -- Dave Abrahams BoostPro Computing http://www.boostpro.com ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] Streamulus v0.1 is out: An EDSL for Event Stream Processing with C++
On 24 Jun 2012, at 22:24, Eric Niebler wrote: > Very interesting! So ... data flow? Or does this take inspiration from > stream databases? > Thank you. Yes, data flow. With central control to make sure things propagate through the graph in topological order. This is necessary for diamond-shaped graphs (think (x+1)/(x+2)). ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] Streamulus v0.1 is out: An EDSL for Event Stream Processing with C++
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/24/2012 8:50 AM, Irit Katriel wrote: > > On 24 Jun 2012, at 03:47, Dave Abrahams wrote: >> >> Well, I think the hello world example is too simple to illustrate >> what this does, and the blog posting is TL;DR, but I skimmed it, >> and still didn't really have a clue. Have you looked at >> http://www.boost.org/doc/libs/1_49_0/doc/html/accumulators.html? >> It seems to have some overlap with the problems you're solving. >> > > I was not aware of accumulators, and I agree that there is some > overlap. As far as I can tell, the main differences are as follows. > Please correct me if I misunderstood anything. > > In the accumulators library, all the accumulators are invoked for > every update to the input. This is why the visitation order can be > determined at compile time. That's correct. > I am building a dependency graph that is used at runtime to > determine which nodes need to be activated, so that inputs only > propagate through the part of the expression that may be affected. > Obviously I am imagining large expressions on many inputs, so that > this is worthwhile doing. Very interesting! So ... data flow? Or does this take inspiration from stream databases? > In addition, I am trying to achieve a more programming-like syntax > for complex expressions, by making the expression (rather than the > accumulator type) encode the dependencies between nodes. Since accumulators is solving a simpler problem, as DSL isn't needed there. For the more general problem you're solving, I think a DSL makes sense. > Example: you want to compute f(g(x),h(x)) over a stream. With > streamulus you define each of the functions f,g,h, and when you > subscribe the expression f(g(x),h(x)), and only then, f learns that > its inputs are the outputs of g and h. With accumulators you would > need to define the accumulator f_of_g_and_h(), make it depend on g > and h, give it a tag, and then add it to the accumulators list. > > Similarly, with streamulus you can define your function f, and then > use it multiple times within the same expression: sqrt(sqrt(x)). > This will build a graph with three nodes : (x--> sqrt-->sqrt) where > x is an input node and the two sqrt's don't need to know or care > about each other because a different instance was constructed for > each node. > > I'm sorry about the too-long blog post.. It mostly deals with the > bad things that can happen if you DIY your stream computations. You > can skip most of it and just look at the definition of the sample > application in the beginning, and then the streamulus solution in > the end. Looks pretty interesting. Thanks for sharing. - -- Eric Niebler BoostPro Computing http://www.boostpro.com -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJP54V6AAoJEAeJsEDfjLbXhKwH/1khe8DP9wSY3rc6KrIfPzux MBSPyiz2vOqT5C/I1pIVXHo71dzNttcLp0NfLTII/++T5Df9MEiE3wfufBBWKDjv elI/2QCgYzEm8IeGUqN3oRz9I0XprsOfWqxsfJs6TglL+JiNlHENBbaH8GGkPI8s Nn1ntI+SWFzhqjaBB9gYyosDYT9f2bRYGQAO0/Ov/+hhjX7tSPc/pET2MOpn0L5p NsyOJv7bqPAICn61yT0O+XJpDBGLeGpqFumsJryMJoxcT/sv1Lvlh2FAIhfrjet6 TAB0lfRKI92O4Bw+kE+lOLRI1TMTKBr8ig+Ykodm0myNLItJRWrV2/9Q2nXTzco= =Ok3v -END PGP SIGNATURE- ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] Streamulus v0.1 is out: An EDSL for Event Stream Processing with C++
On 24 Jun 2012, at 03:47, Dave Abrahams wrote: > > Well, I think the hello world example is too simple to illustrate what > this does, and the blog posting is TL;DR, but I skimmed it, and still > didn't really have a clue. Have you looked at > http://www.boost.org/doc/libs/1_49_0/doc/html/accumulators.html? It > seems to have some overlap with the problems you're solving. > I was not aware of accumulators, and I agree that there is some overlap. As far as I can tell, the main differences are as follows. Please correct me if I misunderstood anything. In the accumulators library, all the accumulators are invoked for every update to the input. This is why the visitation order can be determined at compile time. I am building a dependency graph that is used at runtime to determine which nodes need to be activated, so that inputs only propagate through the part of the expression that may be affected. Obviously I am imagining large expressions on many inputs, so that this is worthwhile doing. In addition, I am trying to achieve a more programming-like syntax for complex expressions, by making the expression (rather than the accumulator type) encode the dependencies between nodes. Example: you want to compute f(g(x),h(x)) over a stream. With streamulus you define each of the functions f,g,h, and when you subscribe the expression f(g(x),h(x)), and only then, f learns that its inputs are the outputs of g and h. With accumulators you would need to define the accumulator f_of_g_and_h(), make it depend on g and h, give it a tag, and then add it to the accumulators list. Similarly, with streamulus you can define your function f, and then use it multiple times within the same expression: sqrt(sqrt(x)). This will build a graph with three nodes : (x--> sqrt-->sqrt) where x is an input node and the two sqrt's don't need to know or care about each other because a different instance was constructed for each node. I'm sorry about the too-long blog post.. It mostly deals with the bad things that can happen if you DIY your stream computations. You can skip most of it and just look at the definition of the sample application in the beginning, and then the streamulus solution in the end. Irit ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] Streamulus v0.1 is out: An EDSL for Event Stream Processing with C++
on Sat Jun 23 2012, Irit Katriel wrote: > Good day, > > Streamulus is a C++ DSEL for event stream processing. > > It uses Proto to transform simple expressions into a data structure that > computes > the expression over an infinite stream of inputs. With user-defined operators > that > can have side effects or maintain state, this scheme is very flexible. I > believe it also > maps well to the way we (or at least I) tend to think about stream > computations. > > V0.1 is the first 'release', and it consists of the basic functionality, > i.e., the language. > Future releases will focus on optimisations. > > See: > > Project web page: http://www.streamulus.com > Blog: http://streamulus.blogspot.co.uk/(The first post explains the > motivation for Streamulus). > > Your feedback will be appreciated. Well, I think the hello world example is too simple to illustrate what this does, and the blog posting is TL;DR, but I skimmed it, and still didn't really have a clue. Have you looked at http://www.boost.org/doc/libs/1_49_0/doc/html/accumulators.html? It seems to have some overlap with the problems you're solving. -- Dave Abrahams BoostPro Computing http://www.boostpro.com ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
