Seems to me that the simplest solution is to create a logical cluster for each app or group of apps that need to be isolated. Using various smaller clusters may be more efficient that using a single large one because there fewer nodes use less network traffic and makes it easier to plan resource allocation. If there is no sharing or dependencies between apps, there is no need to share the cluster. The tools will make it easy to create clusters. Does this make sense for your use case?
-leo On Tue, May 1, 2012 at 12:08 PM, Matthieu Morel <mmo...@apache.org> wrote: > On 4/29/12 2:49 AM, Nan N. Jiang wrote: > >> Hi Team, >> >> I want to start a discuss about ‘Runtime isolation’ for s4. >> >> Currently, in S4 and S4-piper, following S4’s symmetric principal, all >> applications will deployed in all nodes. >> And when starting s4 service, all applications deployed will run in the >> same JVM. >> This can be a disaster. >> Different application will fight for the JVM/hardware resource. >> Bad application even could kill other applications. >> >> Yes. Application Isolation could release the pain. >> But all applications are still fight for the resource in the same node. >> And all the PE in the same Application still fight for JVM resource. >> >> Thinking about a big application which contains hundreds of PE prototypes, >> If we want start this application in one JVM, >> Our s4 cluster must have a set of very high performance machines due to >> the CPU/Memory and other requirement of this application. >> This will limit the usage of S4 in big solution. >> I think the future of s4 cluster will be a big set of normal boxes >> instead of a small set of high performance boxes. >> >> So, compare application isolation, could we go further? >> Could we provide Runtime Isolation function for S4? >> Could we change the local deploy unit from application to a group of PE? >> even one PE? >> If this function can be provided by s4, >> Every PE could run in it’s own JVM if needed. >> Then the resource fighting between PE will be minimum. >> > > Deploying different PEs to different nodes would break the symmetry which > is a key property of the design of S4. However, you are pointing out a real > issue, and one way to provide PE isolation might be at the level of > partitioning. We could provide a way to provide exclusive partitioning for > some PEs using a partition exclusiveness scheme, which would be propagated > to every node. Concretely, messages for PE with key K (or a set of keys) > would be sent to node N, and only messages with such keys would be sent to > node N. This way, we provide PE isolation, without modifying the design of > the platform. > > Do you think this could properly address the isolation issue? > To all: would that harm the overall design of the platform? > > Thanks! > > Matthieu > > > >> Let’s discuss about this. >> >> Regards, >> Jiang Nan >> >> > -- Leo Neumeyer (@leoneu)
