It would be great if the two of you could create jiras for those two
tasks (zk integration, dynamic loading) so that we can track progress
and perhaps have more developers contributing. If you have a good idea
of sub-tasks, also sketch them down to make it easier for people to
jump in and contribute.
-Flavio
On Oct 12, 2011, at 2:42 AM, kishore g wrote:
Hi Henry,
I am working on the using ZK for this. I will create a JIRA and
upload what
I have in mind. But as Leo mentioned we will not be able to have
this for
initial version. And loading/unloading apps can be done independent of
integration with ZK. If we have something working in single node
mode, i
should be able to invoke it in distributed mode when we get notified
by
Zookeeper.
If you are interested in ZK part, we can start a discussion on JIRA.
thanks,
Kishore G
On Tue, Oct 11, 2011 at 3:44 PM, Leo Neumeyer
<[email protected]> wrote:
I was thinking in a roadmap like this::
M1:
- Server starts with no apps.
- Apps are loaded during initialization, for example by searching an
app directory.
- Apps are initialized (no dependencies among apps).
- Apps are started.
M2:
- Add dependencies among apps using EventSource.
M3:
- Add runtime loading/unloading functionality
I think you can start without ZK. I know Kishore is thinking on how
to
use use ZK but I'm not sure it is needed for now at least. Nodes are
symmetric and each node has identical information. All the wiring is
done within the node. I think that OSGI should be able to take care
of
this pretty easily (once you know OSGI). JBOSS Modules should also be
enough but there is no documentation and may require more time than
using OSGI.
Hope you can help with this, if you prepare an initial design we can
iterate and give you feedback. I think you can implement a prototype
for M1. You would need to change Server so it loads apps dynamically.
I was able to run it with Modules that way.
-leo
On Tue, Oct 11, 2011 at 3:22 PM, Henry Saputra <[email protected]
>
wrote:
Hi Leo,
Sorry I miss the Skype meeting. Still looking forward to help.
I would like to work on the dynamic application loading.
I am thinking about having consumer API to help registering an app
as
ZK client but not sure if this is the direction originally in
pipeline.
Anyone else like would like to work on this?
- Henry
On Tue, Oct 11, 2011 at 9:16 AM, Leo Neumeyer <[email protected]
>
wrote:
Skype Meeting Minutes
Time: 2011-10-11 8:00 am Pacific Time
Attendees: Bruce, Matthieu, Karthik, Leo
* Karthik introduced himself, he is currently doing a PhD at
Purdue and
is interested in contributing to the project. He has been working
with
Hadoop and interested in distributed stream processing. He will start
working on the comm layer, perhaps integrating ZK. Bruce will help
him get
started.
* Matthieu: implemented dynamic dispatching. We need to discuss the
processInput() and processOutput() approach to see what makes sense
for the
API. Leo will review and report back and merge.
* Leo added EventSource to support inter app communication and
refactoring comm layer. Also reorganized project into subprojects:
s4-base,
s4-core, s4-comm, s4-example.
* Matthieu and Leo will prototype using Guice for building the
app graph
by subclassing the Guice Configuration class and using a builder
pattern to
create the classes so the application developer doesn't need to use
Guice
directly.
* Bruce: will work on Netty initialization and error recovery.
* People are planning to deploy v0.3 to production. We should try
to
accelerate s4-piper so we can converge and use resources more
efficiently.
* Critical tasks still orphan: dynamic app loading, client adaptor.
* Admin:
* Need to figure out how to use the Apache tool.
* Bruce: let us know if you can do the repo migration this week.
* Matthieu will send instructions on how to set up IRC on Adium
so it
is always on.
--
-leo
flavio
junqueira
research scientist
[email protected]
direct +34 93-183-8828
avinguda diagonal 177, 8th floor, barcelona, 08018, es
phone (408) 349 3300 fax (408) 349 3301
