Hi Henry, I am working on the using ZK for this. I will create a JIRA and upload what I have in mind. But as Leo mentioned we will not be able to have this for initial version. And loading/unloading apps can be done independent of integration with ZK. If we have something working in single node mode, i should be able to invoke it in distributed mode when we get notified by Zookeeper.
If you are interested in ZK part, we can start a discussion on JIRA. thanks, Kishore G On Tue, Oct 11, 2011 at 3:44 PM, Leo Neumeyer <[email protected]> wrote: > I was thinking in a roadmap like this:: > > M1: > - Server starts with no apps. > - Apps are loaded during initialization, for example by searching an > app directory. > - Apps are initialized (no dependencies among apps). > - Apps are started. > > M2: > - Add dependencies among apps using EventSource. > > M3: > - Add runtime loading/unloading functionality > > > I think you can start without ZK. I know Kishore is thinking on how to > use use ZK but I'm not sure it is needed for now at least. Nodes are > symmetric and each node has identical information. All the wiring is > done within the node. I think that OSGI should be able to take care of > this pretty easily (once you know OSGI). JBOSS Modules should also be > enough but there is no documentation and may require more time than > using OSGI. > > Hope you can help with this, if you prepare an initial design we can > iterate and give you feedback. I think you can implement a prototype > for M1. You would need to change Server so it loads apps dynamically. > I was able to run it with Modules that way. > > -leo > > > On Tue, Oct 11, 2011 at 3:22 PM, Henry Saputra <[email protected]> > wrote: > > Hi Leo, > > > > Sorry I miss the Skype meeting. Still looking forward to help. > > > > I would like to work on the dynamic application loading. > > > > I am thinking about having consumer API to help registering an app as > > ZK client but not sure if this is the direction originally in > > pipeline. > > Anyone else like would like to work on this? > > > > - Henry > > > > On Tue, Oct 11, 2011 at 9:16 AM, Leo Neumeyer <[email protected]> > wrote: > >> Skype Meeting Minutes > >> Time: 2011-10-11 8:00 am Pacific Time > >> Attendees: Bruce, Matthieu, Karthik, Leo > >> > >> * Karthik introduced himself, he is currently doing a PhD at Purdue and > is interested in contributing to the project. He has been working with > Hadoop and interested in distributed stream processing. He will start > working on the comm layer, perhaps integrating ZK. Bruce will help him get > started. > >> > >> * Matthieu: implemented dynamic dispatching. We need to discuss the > processInput() and processOutput() approach to see what makes sense for the > API. Leo will review and report back and merge. > >> > >> * Leo added EventSource to support inter app communication and > refactoring comm layer. Also reorganized project into subprojects: s4-base, > s4-core, s4-comm, s4-example. > >> > >> * Matthieu and Leo will prototype using Guice for building the app graph > by subclassing the Guice Configuration class and using a builder pattern to > create the classes so the application developer doesn't need to use Guice > directly. > >> > >> * Bruce: will work on Netty initialization and error recovery. > >> > >> * People are planning to deploy v0.3 to production. We should try to > accelerate s4-piper so we can converge and use resources more efficiently. > >> > >> * Critical tasks still orphan: dynamic app loading, client adaptor. > >> > >> * Admin: > >> * Need to figure out how to use the Apache tool. > >> * Bruce: let us know if you can do the repo migration this week. > >> * Matthieu will send instructions on how to set up IRC on Adium so it > is always on. > >> > >> > >> > >> > > > > > > -- > > -leo >
