Nan, I'm not sure I understand what sharing means. Does it mean that each app gets a copy of the stream or you want each app to get a fraction of the stream? In any case, you seem to need a third app perhaps that feeds copies of the stream.
-Flavio On May 11, 2012, at 11:58 AM, Nan N. Jiang wrote: > Yes. If there is no sharing or dependencies between apps, there is no need > to share the cluster. > > But if there is sharing or dependencies between apps, for example: sharing > event stream(thinking about more then one apps are interested in the same > event stream case), how could s4 solve this issue with logical cluster? > > -- Nan > > On 5/8/12 9:30 PM, "Leo Neumeyer" <leoneume...@gmail.com> wrote: > >> Seems to me that the simplest solution is to create a logical cluster for >> each app or group of apps that need to be isolated. Using various smaller >> clusters may be more efficient that using a single large one because there >> fewer nodes use less network traffic and makes it easier to plan resource >> allocation. If there is no sharing or dependencies between apps, there is >> no need to share the cluster. The tools will make it easy to create >> clusters. Does this make sense for your use case? >> >> -leo >> >> On Tue, May 1, 2012 at 12:08 PM, Matthieu Morel <mmo...@apache.org> wrote: >> >>> On 4/29/12 2:49 AM, Nan N. Jiang wrote: >>> >>>> Hi Team, >>>> >>>> I want to start a discuss about ŒRuntime isolation¹ for s4. >>>> >>>> Currently, in S4 and S4-piper, following S4¹s symmetric principal, all >>>> applications will deployed in all nodes. >>>> And when starting s4 service, all applications deployed will run in the >>>> same JVM. >>>> This can be a disaster. >>>> Different application will fight for the JVM/hardware resource. >>>> Bad application even could kill other applications. >>>> >>>> Yes. Application Isolation could release the pain. >>>> But all applications are still fight for the resource in the same node. >>>> And all the PE in the same Application still fight for JVM resource. >>>> >>>> Thinking about a big application which contains hundreds of PE prototypes, >>>> If we want start this application in one JVM, >>>> Our s4 cluster must have a set of very high performance machines due to >>>> the CPU/Memory and other requirement of this application. >>>> This will limit the usage of S4 in big solution. >>>> I think the future of s4 cluster will be a big set of normal boxes >>>> instead of a small set of high performance boxes. >>>> >>>> So, compare application isolation, could we go further? >>>> Could we provide Runtime Isolation function for S4? >>>> Could we change the local deploy unit from application to a group of PE? >>>> even one PE? >>>> If this function can be provided by s4, >>>> Every PE could run in it¹s own JVM if needed. >>>> Then the resource fighting between PE will be minimum. >>>> >>> >>> Deploying different PEs to different nodes would break the symmetry which >>> is a key property of the design of S4. However, you are pointing out a real >>> issue, and one way to provide PE isolation might be at the level of >>> partitioning. We could provide a way to provide exclusive partitioning for >>> some PEs using a partition exclusiveness scheme, which would be propagated >>> to every node. Concretely, messages for PE with key K (or a set of keys) >>> would be sent to node N, and only messages with such keys would be sent to >>> node N. This way, we provide PE isolation, without modifying the design of >>> the platform. >>> >>> Do you think this could properly address the isolation issue? >>> To all: would that harm the overall design of the platform? >>> >>> Thanks! >>> >>> Matthieu >>> >>> >>> >>>> Let¹s discuss about this. >>>> >>>> Regards, >>>> Jiang Nan >>>> >>>> >>> >> > flavio junqueira senior research scientist f...@yahoo-inc.com direct +34 93-183-8828 avinguda diagonal 177, 8th floor, barcelona, 08018, es phone (408) 349 3300 fax (408) 349 3301
