Re: [Architecture] [Dev] Tomcat Session Replication Using Caching
On Thu, Oct 2, 2014 at 9:22 AM, Srinath Perera srin...@wso2.com wrote: I am not sure from notes what we decided. Cannot we just used hazelcast session replication? does above design you mentioned built on top of Hazelcast? Yes, the plan is to use what Hazelcast provides. Otherwise, we will anyway have to solve all the problems the Hz implementation has solved. --Srinath On Wed, Oct 1, 2014 at 2:18 PM, Pubudu Dissanayake pubu...@wso2.com wrote: Hi All, Based on the Architecture review of Tomcat session replication using caching , following aspects were discussed. Participants - Azeez, Sameera , Aruna, Jayanga, Gayashan, SupunM, Niranjan and Myself Proposed solution. - key-val: (CarbonSessionCacheKey, CarbonSessionCache) - Initial implementation was based on javax.caching api's. I've created CarbonSessionCache and CarbonSessionCacheKey to maintain HttpSession in the Hazelcast distributed map. Session will be intercepted by using a Tomcat Valve. - *Verdict* on the proposed solution. - Cache map Serialization and Deserialization is very expensive operation due to heavy traffic. Therefore it is not efficient to replicate the whole session object in the cache map. Counter Proposed solution - Having a Hazelcast map per session object. In that way if an attribute in the session changes the whole entry should not have to be replicated, only the changed attribute. - Have a per-session cache - ( session_id ). - Issue in this approach as follows - How can we acquire the session object from the map ? *Final verdict *on the feature implementation - Instead of using the caching APIs, This needs to fix in Hazelcast level and do it in the optimal way. [1] - HazelcastWM http://docs.hazelcast.org/docs/1.9.4/manual/html/ch08.html [2], Bring in the filter code level in to the tomcat valve instead -Pubudu [1] http://hazelcast.com/use-cases/web-session-clustering/ [2] http://docs.hazelcast.org/docs/1.9.4/manual/html/ch08.html On Wed, Oct 1, 2014 at 10:51 AM, Pubudu Dissanayake pubu...@wso2.com wrote: Hi Srinath Hazelcast WM allows us to cluster user http sessions automatically. Followings are required for enabling Hazelcast Session Clustering http://hazelcast.com/use-cases/web-session-clustering/ [2][3] - Target application or web server should support Java 1.5+ - Target application or web server should support Servlet 2.4+ spec - Session objects that needs to be clustered have to be Serializable Purpose of web session clustering is to replicate web session state across a distributed application servers. Benefits, allow us to scale out but also avoid any single point of failure and allow us to dynamically handle node failure within the application tier. performance of this web session clustering is high according to VP Hazelcast https://www.youtube.com/watch?v=8a0LrlOIXq0. Using Hazelcast session replication can be achieved in two different ways. Observations as follows. - *Hazelcast Enterprise* has native web session clustering build in to Apache Tomcat. - *Opensource Hazelcast* has provide a web session clustering as an external filter, basically Hazelcast filter has to be placed before all of other filters and that will enable us to provide the web session clustering in opensource. In this use case it doesn't required changes to application itself. [1] https://redmine.wso2.com/issues/2816 [2] http://hazelcast.com/use-cases/web-session-clustering/ [3] http://hazelcast.org/docs/latest/manual/html/httpsessionclustering.html [4] https://www.youtube.com/watch?v=8a0LrlOIXq0 More info $subject - Web Session Clustering / Session Replication using Hazelcast -Pubudu On Wed, Oct 1, 2014 at 10:41 AM, Srinath Perera srin...@wso2.com wrote: We need to use Hazelcast When implemented we need to test this with multilple nodes (at least 4) and in high load conditions before it is done done. --Srinath On Wed, Oct 1, 2014 at 8:52 AM, Dinesh J Weerakkody dine...@wso2.com wrote: Hi, I came across this project [1] and might be useful. It is based on memcached library. [1] http://code.google.com/p/memcached-session-manager/ On Tue, Sep 30, 2014 at 8:01 PM, Pubudu Dissanayake pubu...@wso2.com wrote: Hi All, I am working on an improvement for Carbon to enable tomcat session replication support using caching .Our previous implementation was based on a Tomcat Valve ( CarbonTomcatSessionReplicationValve.java).And it was used to replicate the session and adding it to cluster message. Http Session replication in carbon works like this (its quite similar to how the same work in a standalone tomcat). 1. Not every webbapp session will be replicated. You need to set a context property (distributable) for the webapp you need. 2. Once enabled, and when the session is modified on a node, that modification is sent to other
Re: [Architecture] [Siddhi] Extension Support
Hi, This is a suggestion, I think its useful to have a version with the siddhi extension name, The same extension can exists with different versions. Idea is if extension get change and new version arrives, no need to change existing queries. Ex -: currency_1.0.siddhiext In the query should call the extension with the version. Then currency new extension arrives it may be 2.0. Will this be useful? Thanks On Tue, Sep 30, 2014 at 5:52 PM, Sachini Jayasekara sachi...@wso2.com wrote: Hi all, New Siddhi implementation supports custom functions and custom output aggregators. Custom functions can be written extending the FunctionExecutor class. Custom output aggregators can be written implementing AttributeAggregator interface. In earlier version, to write a custom output aggregator, user had to implement both OutputAttributeAggregatorFactory and OutputAttributeAggregator interfaces. (Now there is no need to implement 2 interfaces user can simply add the logic to one class which implements AttributeAggregator interface). Then the implemented custom classes has to be compiled and the jar files should be added to the class path. User then has to add the custom function name and fully-qualified class name of the custom extension to a file named namespace of the function/aggregator . siddhiext. (eg:- if namespace of a function/aggregator is currency function/aggregator name and fully-qualified class name should be added to a file named currency.siddhiext ). For each extension .siddhiext should be created. This file should also be added to class path. In earlier version, jars and siddhiext files have to be added to specific locations but now user can and those to anywhere in class path. Extension loading process of Siddhi will search for files ending with .siddhiext in class path and add them as extensions. User can refer to the custom classes giving the namespace and function name. Thanks -- *Thanks Regards,Sachini JayasekaraSoftware Engineer; **WSO2 Inc. * *lean . enterprise . middleware | http://wso2.com http://wso2.com * ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manoj Gunawardena Tech Lead WSO2, Inc.: http://wso2.com lean.enterprise.middleware Mobile : +94 77 2291643 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Siddhi] Extension Support
Its a good addition @Sachini can you create a jira for this? We'll add this when we have stabilised the basic functionalities. On Thu, Oct 2, 2014 at 3:26 PM, Manoj Gunawardena man...@wso2.com wrote: Hi, This is a suggestion, I think its useful to have a version with the siddhi extension name, The same extension can exists with different versions. Idea is if extension get change and new version arrives, no need to change existing queries. Ex -: currency_1.0.siddhiext In the query should call the extension with the version. Then currency new extension arrives it may be 2.0. Will this be useful? Thanks On Tue, Sep 30, 2014 at 5:52 PM, Sachini Jayasekara sachi...@wso2.com wrote: Hi all, New Siddhi implementation supports custom functions and custom output aggregators. Custom functions can be written extending the FunctionExecutor class. Custom output aggregators can be written implementing AttributeAggregator interface. In earlier version, to write a custom output aggregator, user had to implement both OutputAttributeAggregatorFactory and OutputAttributeAggregator interfaces. (Now there is no need to implement 2 interfaces user can simply add the logic to one class which implements AttributeAggregator interface). Then the implemented custom classes has to be compiled and the jar files should be added to the class path. User then has to add the custom function name and fully-qualified class name of the custom extension to a file named namespace of the function/aggregator . siddhiext. (eg:- if namespace of a function/aggregator is currency function/aggregator name and fully-qualified class name should be added to a file named currency.siddhiext ). For each extension .siddhiext should be created. This file should also be added to class path. In earlier version, jars and siddhiext files have to be added to specific locations but now user can and those to anywhere in class path. Extension loading process of Siddhi will search for files ending with .siddhiext in class path and add them as extensions. User can refer to the custom classes giving the namespace and function name. Thanks -- *Thanks Regards,Sachini JayasekaraSoftware Engineer; **WSO2 Inc. * *lean . enterprise . middleware | http://wso2.com http://wso2.com * ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manoj Gunawardena Tech Lead WSO2, Inc.: http://wso2.com lean.enterprise.middleware Mobile : +94 77 2291643 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *S. Suhothayan* Technical Lead Team Lead of WSO2 Complex Event Processor *WSO2 Inc. *http://wso2.com * http://wso2.com/* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ http://suhothayan.blogspot.com/twitter: http://twitter.com/suhothayan http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan http://lk.linkedin.com/in/suhothayan* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Siddhi] Extension Support
Hi all, Created a jira at [1]. [1]https://wso2.org/jira/browse/CEP-964 Thanks Sachini On Thu, Oct 2, 2014 at 4:30 PM, Sriskandarajah Suhothayan s...@wso2.com wrote: Its a good addition @Sachini can you create a jira for this? We'll add this when we have stabilised the basic functionalities. On Thu, Oct 2, 2014 at 3:26 PM, Manoj Gunawardena man...@wso2.com wrote: Hi, This is a suggestion, I think its useful to have a version with the siddhi extension name, The same extension can exists with different versions. Idea is if extension get change and new version arrives, no need to change existing queries. Ex -: currency_1.0.siddhiext In the query should call the extension with the version. Then currency new extension arrives it may be 2.0. Will this be useful? Thanks On Tue, Sep 30, 2014 at 5:52 PM, Sachini Jayasekara sachi...@wso2.com wrote: Hi all, New Siddhi implementation supports custom functions and custom output aggregators. Custom functions can be written extending the FunctionExecutor class. Custom output aggregators can be written implementing AttributeAggregator interface. In earlier version, to write a custom output aggregator, user had to implement both OutputAttributeAggregatorFactory and OutputAttributeAggregator interfaces. (Now there is no need to implement 2 interfaces user can simply add the logic to one class which implements AttributeAggregator interface). Then the implemented custom classes has to be compiled and the jar files should be added to the class path. User then has to add the custom function name and fully-qualified class name of the custom extension to a file named namespace of the function/aggregator . siddhiext. (eg:- if namespace of a function/aggregator is currency function/aggregator name and fully-qualified class name should be added to a file named currency.siddhiext ). For each extension .siddhiext should be created. This file should also be added to class path. In earlier version, jars and siddhiext files have to be added to specific locations but now user can and those to anywhere in class path. Extension loading process of Siddhi will search for files ending with .siddhiext in class path and add them as extensions. User can refer to the custom classes giving the namespace and function name. Thanks -- *Thanks Regards,Sachini JayasekaraSoftware Engineer; **WSO2 Inc. * *lean . enterprise . middleware | http://wso2.com http://wso2.com * ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manoj Gunawardena Tech Lead WSO2, Inc.: http://wso2.com lean.enterprise.middleware Mobile : +94 77 2291643 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *S. Suhothayan* Technical Lead Team Lead of WSO2 Complex Event Processor *WSO2 Inc. *http://wso2.com * http://wso2.com/* lean . enterprise . middleware *cell: (+94) 779 756 757 %28%2B94%29%20779%20756%20757 | blog: http://suhothayan.blogspot.com/ http://suhothayan.blogspot.com/twitter: http://twitter.com/suhothayan http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan http://lk.linkedin.com/in/suhothayan* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Thanks Regards,Sachini JayasekaraSoftware Engineer; **WSO2 Inc. * *lean . enterprise . middleware | http://wso2.com http://wso2.com * ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] API Manager - Self Signup for tenant's API Store
Perfect! Thanks Chamila! Dmitry On Wed, Oct 1, 2014 at 8:33 PM, Chamila Adhikarinayake chami...@wso2.com wrote: Hi Dmitry, This feature is already done and we ship this with APIM 1.8. Sorry for not updating this thread. Regarding the scenarios, We have provided facility to add custom workflow extension. Currently we ship UserSignUpWSWorkflowExecutor which also has a user approval process [1]. We can also create custom workflow extension to suit the scenario mentioned and plug them in. We can also deploy scenario as a business process in WSO2 BPS and connect using our UserSignUpWSWorkflowExecutor. [1] https://docs.wso2.com/display/AM170/Adding+a+User+Signup+Workflow Thanks, Chamila. On Thu, Oct 2, 2014 at 3:05 AM, Dmitry Sotnikov dmi...@wso2.com wrote: Any update on that? What is the ETA for the feature? Is there something we can review yet? Were you able to satisfy all the end-user experience requirements? Dmitry On Fri, Sep 5, 2014 at 10:09 AM, Dmitry Sotnikov dmi...@wso2.com wrote: Thanks Chamila! Here is the workflow that I would like to see in the API Cloud: Scenario A: User Self Sign-Up: 1. Plato is an app developer and wants to develop an app for the population of the Atlantis island to track the water level and notify when the island goes under the sea. He finds that gods have set up a website with the developer program: apis.atlantisisland.gr and goes to the site to read about the APIs. 2. When browsing the API Store at some point Plato tries to access functionality which requires authentication (e.g. Subscribe to an API), 3. Plato is presented to choice to log in or sign-up, 4. If Plato clicks Sign Up, he is asked provide his email address. 5. Plato provides his gmail address, 6. He gets an email inviting him to join the Atlatis developer program with a one-time link that takes him to the Store, asks him to specify and confirm new password. 7. Plato is now logged into the Atlastic API Store, and can perform all activity there. 8. Plato has Subscriber role - so he cannot actually go to Atlatis API Publisher, etc. Possible variations of that: Scenario B: Approval is required: Zeus is the administrator of the API program and gets a request to approve Plato's membership. In this case, this needs to be properly communicated to both Zeus and Plato, so they know what is going on, what is expected of them, current status, etc. Scenario C: Zeus actually wants to invite Plato to the development program: Zeus goes to the corresponding UI, provides Plato's email address, and Plato receives email with the invitation and one-time link (obviously approvals are not required in this case.) Scenario D: 1. Homer decides to also write an Atlatis app, and wants to sign-up. 2. When he tries to sign-up he gets notified that he already has an account from WSO2 which he used for his Trojan app and which he can reuse. All the emails that get sent need to be brandable by the tenant administrators so when Troy have their API program, Trojan emails look different from the ones for Atlantis. I understand that some of these steps will be different in the cloud and in on-premise API Manager because of the custom authentication. You will need to discuss with the cloud team to make sure that the implementation is compatible, etc. I hope this helps. :) Dmitry On Thu, Sep 4, 2014 at 11:09 PM, Chamila Adhikarinayake chami...@wso2.com wrote: looping Dmitry to the thread On Fri, Sep 5, 2014 at 10:26 AM, Chamila Adhikarinayake chami...@wso2.com wrote: Hi all, The current user signup method (jsFunction_addUser() [1]) in API manager uses addUser method in UserRegistrationService[2] (through UserRegistrationAdminService from APIM) to register the user and suggested to use that same method for tenant user signup as well (Shariq has modified this so that tenant-wise roles can be taken from the registry and assign them to the tenant user ). But with that modification, This method cannot be used for tenant signup with the current user signup workflow in the api manager. When signing up an user to super user store, first add a user by calling adduser method (from this method, user is assigned with default internal/identity role from UserRegistrationService) and then follow the signup workflow and finally add the role to that user. In the signup workflow, user approval process is managed and till then the user cannot log in. But when addUser method is called for tenant signup, a tenant user is created and all his roles are assigned to him at the creation point. As a result, second point (approval process) cannot be done. (user can log in before the approval process is done). As a result, method to assign roles by using '/_system/governance/repository/identity/sign-up-config' (mentioned in 'User roles for tenants' in my first mail ) cannot be used in this case. *Modified method to add user roles for tenants* As a
