Hi Azeez, On Tue, May 22, 2012 at 2:27 PM, Afkham Azeez <[email protected]> wrote:
> > > On Tue, May 22, 2012 at 2:22 PM, Anjana Fernando <[email protected]> wrote: > >> Hi Dimuthu, >> >> On Tue, May 22, 2012 at 2:15 PM, Dimuthu Leelarathne >> <[email protected]>wrote: >> >>> Hi, >>> >>> Currently org.wso2.carbon.ntask.core.service.impl.TaskServiceImpl is >>> initializing task managers for all tenants at startup. This is against our >>> tenant lazy loading concepts. >>> >>> Please move tenant ntask initialization to a >>> Axis2ConfigurationContextObserver. This will be called when >>> TenanConfiguraitonContext is loaded. >>> >> >> As I remember, this was already brought up by Azeez earlier. The >> rationale is, a scheduled task is not a service. It is not something that >> is activated on-demand by a user in a specific tenant. But is suppose to >> work in the scheduled manner continuously. So when a user in a tenant >> schedules a task, it should always run, and should not unload when the >> tenant is unloaded. And when starting the server also, all the tasks should >> continue. >> > > But now we have a major problem if we have say 100,001 tenants. We will > create this for all tenants unnecessarily. With the next release, we have > tenant partitions defined at the LB level, which relies on the worker nodes > to lazy load the tenants. There is no tenant partitioning concept at the > worker node level. So, even though cluster as-0-100 loads tenants 0-100, > the task thing will load all 100,001! We need to find a solution to this > issue. > OK, so this has actually turned into a tenant partitioning problem. First of all, when you say initializing a TaskManager, it just means, creating an object to represent a tasks for a specific tenant and a connection to a centrally managed Group object in ZooKeeper. If the tasks are not scheduled, it will simply take a block of memory and just sit there. And it will not load the tenants, I'm simply retrieving the total tenant id list and only loading the tenant aware task managers, it should not load other services and all. And also, the actual tasks will not be scheduled in a single server, they will be distributed across the whole cluster. So anyway, to support the tenant partitioning problem. I suggest the following. Lets say we have several clusters to represent different tenant partitions. For example, 3 clusters, to divide the tenants equally. When starting the server, we can set a Java system property saying, the number of clusters used for tenant partitioning and the partitioning cluster the current server will belong to. So for example -Dtenant.partition.count=3 -Dmy.tenant.partition=2 (suggest suitable names). So with these information at hand, when starting the server, we can start only the tasks suppose to be scheduled in this server (take tenant_id % tenant.partition.count and match it to my.tenant.partition), and also when a new tenant is joining, we can do the same. Hopefully, this type of a method can be used for other type of similar services too. I can implement this in ntask now, if it is an acceptable solution. Cheers, Anjana. >> Cheers, >> Anjana. >> >> >>> >>> thanks, >>> dimuthu >>> >>> >>> >>> -- >>> Dimuthu Leelarathne >>> Technical Lead >>> >>> WSO2, Inc. (http://wso2.com) >>> email: [email protected] >>> >>> Lean . Enterprise . Middleware >>> >>> >> >> >> -- >> *Anjana Fernando* >> Associate Technical Lead >> WSO2 Inc. | http://wso2.com >> lean . enterprise . middleware >> >> _______________________________________________ >> Dev mailing list >> [email protected] >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > *Afkham Azeez* > Director of Architecture; WSO2, Inc.; http://wso2.com > Member; Apache Software Foundation; http://www.apache.org/ > * <http://www.apache.org/>** > email: **[email protected]* <[email protected]>* cell: +94 77 3320919 > blog: **http://blog.afkham.org* <http://blog.afkham.org>* > twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> > * > linked-in: **http://lk.linkedin.com/in/afkhamazeez* > > * > * > *Lean . Enterprise . Middleware* > > -- *Anjana Fernando* Associate Technical Lead WSO2 Inc. | http://wso2.com lean . enterprise . middleware
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
