Hi Dimuthu,
On Tue, May 22, 2012 at 3:12 PM, Dimuthu Leelarathne <[email protected]>wrote:
> Hi Anjana,
>
> Say there are 2000 tenants.
>
> This is just one call, but after that you load all 2000 user realms in a
> iterator for each tenant. Meaning another 2000 db calls to check for custom
> realms. Now it is 2001 db calls.
>
I'm sorry, I didn't understand what "custom realms" are. So the code I use
is like the following,
Tenant[] tenants = TasksDSComponent.getRealmService().getTenantManager().
getAllTenants();
for (Tenant tenant : tenants) {
tids.add(tenant.getId());
}
So you mean, when I do "tenant.getId()" also, there is another database
call going out? ..
Cheers,
Anjana.
> thanks,
> dimuthu
>
>
>
>> Cheers,
>> Anjana.
>>
>>
>>>
>>>
>>> On Tue, May 22, 2012 at 2:59 PM, Anjana Fernando <[email protected]>wrote:
>>>
>>>> Hi Azeez,
>>>>
>>>> On Tue, May 22, 2012 at 2:27 PM, Afkham Azeez <[email protected]> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, May 22, 2012 at 2:22 PM, Anjana Fernando <[email protected]>wrote:
>>>>>
>>>>>> Hi Dimuthu,
>>>>>>
>>>>>> On Tue, May 22, 2012 at 2:15 PM, Dimuthu Leelarathne <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Currently org.wso2.carbon.ntask.core.service.impl.TaskServiceImpl is
>>>>>>> initializing task managers for all tenants at startup. This is against
>>>>>>> our
>>>>>>> tenant lazy loading concepts.
>>>>>>>
>>>>>>> Please move tenant ntask initialization to a
>>>>>>> Axis2ConfigurationContextObserver. This will be called when
>>>>>>> TenanConfiguraitonContext is loaded.
>>>>>>>
>>>>>>
>>>>>> As I remember, this was already brought up by Azeez earlier. The
>>>>>> rationale is, a scheduled task is not a service. It is not something that
>>>>>> is activated on-demand by a user in a specific tenant. But is suppose to
>>>>>> work in the scheduled manner continuously. So when a user in a tenant
>>>>>> schedules a task, it should always run, and should not unload when the
>>>>>> tenant is unloaded. And when starting the server also, all the tasks
>>>>>> should
>>>>>> continue.
>>>>>>
>>>>>
>>>>> But now we have a major problem if we have say 100,001 tenants. We
>>>>> will create this for all tenants unnecessarily. With the next release, we
>>>>> have tenant partitions defined at the LB level, which relies on the worker
>>>>> nodes to lazy load the tenants. There is no tenant partitioning concept at
>>>>> the worker node level. So, even though cluster as-0-100 loads tenants
>>>>> 0-100, the task thing will load all 100,001! We need to find a solution to
>>>>> this issue.
>>>>>
>>>>
>>>> OK, so this has actually turned into a tenant partitioning problem.
>>>> First of all, when you say initializing a TaskManager, it just means,
>>>> creating an object to represent a tasks for a specific tenant and a
>>>> connection to a centrally managed Group object in ZooKeeper. If the tasks
>>>> are not scheduled, it will simply take a block of memory and just sit
>>>> there. And it will not load the tenants, I'm simply retrieving the total
>>>> tenant id list and only loading the tenant aware task managers, it should
>>>> not load other services and all. And also, the actual tasks will not be
>>>> scheduled in a single server, they will be distributed across the whole
>>>> cluster.
>>>>
>>>> So anyway, to support the tenant partitioning problem. I suggest the
>>>> following. Lets say we have several clusters to represent different tenant
>>>> partitions. For example, 3 clusters, to divide the tenants equally. When
>>>> starting the server, we can set a Java system property saying, the number
>>>> of clusters used for tenant partitioning and the partitioning cluster the
>>>> current server will belong to. So for example -Dtenant.partition.count=3
>>>> -Dmy.tenant.partition=2 (suggest suitable names). So with these information
>>>> at hand, when starting the server, we can start only the tasks suppose to
>>>> be scheduled in this server (take tenant_id % tenant.partition.count and
>>>> match it to my.tenant.partition), and also when a new tenant is joining, we
>>>> can do the same. Hopefully, this type of a method can be used for other
>>>> type of similar services too. I can implement this in ntask now, if it is
>>>> an acceptable solution.
>>>>
>>>> Cheers,
>>>> Anjana.
>>>>
>>>>
>>>>>> Cheers,
>>>>>> Anjana.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> thanks,
>>>>>>> dimuthu
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Dimuthu Leelarathne
>>>>>>> Technical Lead
>>>>>>>
>>>>>>> WSO2, Inc. (http://wso2.com)
>>>>>>> email: [email protected]
>>>>>>>
>>>>>>> Lean . Enterprise . Middleware
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Anjana Fernando*
>>>>>> Associate Technical Lead
>>>>>> WSO2 Inc. | http://wso2.com
>>>>>> lean . enterprise . middleware
>>>>>>
>>>>>> _______________________________________________
>>>>>> Dev mailing list
>>>>>> [email protected]
>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Afkham Azeez*
>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>>>> Member; Apache Software Foundation; http://www.apache.org/
>>>>> * <http://www.apache.org/>**
>>>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919
>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>*
>>>>> twitter:
>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez>
>>>>> *
>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez*
>>>>>
>>>>> *
>>>>> *
>>>>> *Lean . Enterprise . Middleware*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Anjana Fernando*
>>>> Associate Technical Lead
>>>> WSO2 Inc. | http://wso2.com
>>>> lean . enterprise . middleware
>>>>
>>>
>>>
>>>
>>> --
>>> *Afkham Azeez*
>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>> Member; Apache Software Foundation; http://www.apache.org/
>>> * <http://www.apache.org/>**
>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919
>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>*
>>> twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez>
>>> *
>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez*
>>> *
>>> *
>>> *Lean . Enterprise . Middleware*
>>>
>>>
>>
>>
>> --
>> *Anjana Fernando*
>> Associate Technical Lead
>> WSO2 Inc. | http://wso2.com
>> lean . enterprise . middleware
>>
>
>
>
> --
> Dimuthu Leelarathne
> Technical Lead
>
> WSO2, Inc. (http://wso2.com)
> email: [email protected]
>
> Lean . Enterprise . Middleware
>
>
--
*Anjana Fernando*
Associate Technical Lead
WSO2 Inc. | http://wso2.com
lean . enterprise . middleware
_______________________________________________
Dev mailing list
[email protected]
http://wso2.org/cgi-bin/mailman/listinfo/dev
