Hello Mark, all, On 31 Dec 2010, at 10:04 , Mark Machielsen wrote:
> In the last conference call we concluded that some services will be tenant > aware. For example the Cassandra bundle won't we deployed in every tenant > container because this would not scale. So we now have two options: a tenant > aware bundle, and a bundle which is not tenant aware and runs in every tenant > container. Agreed, in even more generic terms you could say that there are components that provide services (if you need more than one, you need to deploy multiple components) and components that provide service factories (which is our tenant aware option). Then there are different deployment options in the sense that you can deploy all components in the same container, or in different ones. We concluded that if a component is not tenant aware it needs to be deployed in a separate container per tenant. If it is, it can be deployed in one container and, via remote services, be published in each tenant specific container. > Question which arise: > - when to use which option When the savings in resource consumption are worth it, create a multi tenant aware component. In theory, we never need them, if each tenant gets its own container. However, we might not be using the available resources (memory, threads, ports, ...) effectively if we do it like that. At the container level that's why we're considering running multiple JVMs and multiple containers per JVM. At the component level, multi tenancy can help us pool resources. The downsides? Added complexity and individual service invocations will be slower. That last point is important, as normal service calls will be direct method invocations and these will be remote service calls. Even with an efficient implementation, those calls will be significantly slower so any gains of going for a multi tenant aware implementation should be big enough to counteract that. > - what are we going to communicate: is the default tenant aware or not The default, and easiest, is to not make your component tentant aware. > - is it explainable that there are two ways of developing I think so. And if you can't explain it to someone, just tell them there's only one way. ;) > - we should prevent that a bundle is developed as tenant unaware and must be > transformed to tenant aware because of scaling issues. Or worse: a tenant > unaware bundle can be used in a certain application, but not in another > application, because of a different deployment So when in general do you resolve performance issues? A lot of the time they are hard to predict. Of course that does not mean you cannot properly design a component, use scalable algorithms, etc. I don't think there is a generic recipe though. > - when we are building an application where all tenants are the same (for > example BC), why should we develop tenant unaware bundles (besides the > programming model) See above (downsides). However, if you deploy all tenant aware bundles in the same container and there effectively is no "container per tenant" then one of the downsides (slower invocations) can be avoided. If the added complexity is no issue, and the lack of isolation is not either, I see no reason not to do it like that. As a side note, if you're running an application for tenants A, B and C and suddenly A starts consuming a lot of resources, a cluster manager might decide to migrate A to a different container. Without trying to start a full thread in this discussion about how to migrate components, migrating some tenants of a multi tenant aware component might also be more difficult and you have to take that into account as well. > @Bram: for the cons you state "I got OOM at 500 tenant with 256M memory in > poc setup": what causes this OOM? It seems to me as a pretty simple > application which only 500 tenants (and sure, only 256M memory). It is the > overhead of 500 containers or something else? My guess would be it is, but you can quite easily run a benchmark that instantiates and starts 500 completely empty containers and measure memory consumption. Only half a megabyte of memory for a container does not sound too bad. > What would be the memory footprint for BC assuming all BC specific bundles > are tenant unaware, assuming BC also needs memory for caching purposes. I'll leave that one to Bram, but in general I think it's a hard question to answer because I cannot imagine that it's independent of how BC is being used. Unless you're talking about code size, not runtime consumption. > How well does it scale when I have a small application (eg openID server) > with lots of tenants. You'll have the overhead of each OSGi framework instance (which you can measure as mentioned above). Other than that, there should not be much overhead if each tenant uses its own data. I mean you might be able to more effectively share resources if you have a multi tenant aware implementation (one pool for every shared resource instead of lots of smaller pools) but that's about it. > A more practical question: assume I have a tenant unaware service with a REST > interface. I assume the REST is received by the tenant aware HTTP service > (which listens to a certain port) in the tenant aware container. How is this > call forwarded to the tenant unaware service? I don't think we currently have > a mechanism to go from tenant aware to tenant unaware, where you need to know > in what container the tenant lives. So far we only talked about OSGi services, and there we use remote services to publish the right tenant service into its container. For REST services, there is no such thing as a service registry, so we cannot discover them. That means we need to somehow configure them. In this case it means we need to map each tenant onto the appropriate REST URL. Greetings, Marcel
