Re: [Architecture] About EMM-Android Local Push Notification Mechanism
Hi Kasun, I think we must implement the method you are proposing. Since this AlarmManager system service will wakeup our EMM agent in every 10 mins (or in configured intervals) it will make sure that our EMM agent is running in the device. Otherwise there would be no way that we could guarantee that EMM-agent process is running in the device. Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Wed, Apr 9, 2014 at 9:33 PM, Kasun Dananjaya Delgolla kas...@wso2.comwrote: Hi All, This is to brief and discuss about the architecture of the $subject. So here's how it works. We gonna enable both GCM and Local push as switchable options so that depending on the requirement, we can choose which method to go with. Android's native service such as AlarmManager will be used to trigger local notifications so that it will be very effective with Battery consumption. On Agent launch, if the selected method is local push, Agent will initiate a repeating event using AlarmManager. (Ex: Repeat every 10mins). When the AlarmManager triggers the event, it simply get captured by the Local Notification Receiver resides in our EMM Agent. And then it will perform the necessary action (Operation/Contacting the server). I have already done a thorough testing on battery consumption using this method which is less than 0.5% when this service is running every 5 minutes with a web service call. Therefore this can actually be used as an alternative to GCM. And it can also be used to capture violations (If the device is stolen and the network is switch off etc) and implement a self initiation protocol in such cases. But IMO we should keep both the options because those 2 can come handy in different situations. So WDYT? Given below is how Android EMM Agent will look like, [image: Inline image 1] If you want an in-depth diagram on the current EMM Agent Architecture [1]. [1] - https://docs.google.com/a/wso2.com/drawings/d/12LOSMJxPtcI_XXjkD5RnPp-uaJGTFtCC-MXkmPLT4LY/edit?usp=sharing Thanks -- Kasun Dananjaya Delgolla Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware Tel: +94 11 214 5345 Fax: +94 11 2145300 Mob: + 94 777 997 850 Blog: http://kddcodingparadise.blogspot.com Linkedin: *http://lk.linkedin.com/in/kasundananjaya http://lk.linkedin.com/in/kasundananjaya* inline: Local notifications - Android.png___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] EMM - UI Design
Hi, It's true that we should have unified UI's for all the applications. But reinventing the front-end MVC frameworks won't be a good idea (agree with Chan's idea). IMO we should look deeper into this and find a proper way to do the front-ends of our application (at least selecting the technologies setting-up guidelines for UI design will be helpful). Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Mon, May 5, 2014 at 9:44 AM, Pubudu Dissanayake pubu...@wso2.com wrote: Please refer my in-line comment. On Sun, May 4, 2014 at 9:29 AM, Chanaka Jayasena chan...@wso2.com wrote: I agree that we need to unify the look and feel of these UIs. When working with WSO2 Cloud the importance of unified look and feel is more visible. Considering the top and left menus usability wise both have pros and cons. Generally it's good to have a left side menu when there are growing number of options for it. In the Cloud UI it's better to have the menu at the top, since we have to inject the same menu to the App Factory UI and API Store/Publisher UIs. @Dilshan, API Store, API Publisher, and App Factory is done with an older version of Caramel. This is why there is a significant difference in the coding pattern. I don't think Caramel existed when the API Store/Manger applications were started developing. The latest version of Caremel we use for Stratos Web Console is way better than the older versions. It pushes front end developers to write clean code without messing with complex jQuery Dom manipulation to present data to the UI. We have handlebars.js with Caramel. Even though handebars.js ( Template Engine) and AngularJS ( HTML Compiler and data binder ) is completely different things, using them together doesn't make sense IMO. Chthura please do a research and find out that his need for AngularJS is filled by handebars.js or not. To have a consistent look and feel in Carbon products, the way Carbon UI initially developed helped. If we can have a similar patten where there is a core component with default styles, layout and let each product override these styles with there own, it will drive every product to have a common look and feel. Indeed, Having unified UI framework is a must and let products to build their own UI is a bad idea. We should come up with a proper structure ( previous carbon releases has this component.xml based approach ) therefore we can build on top of it (hence can re-use it for future carbon UI framework) . IMO consistency has to be there. thanks, Chanaka -- *Pubudu Dissanayake* Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware Mobile: 0775503304 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] Writing Unit tests for iOS EMM client
Hi, I have started $subject for WSO2 EMM next release. In the iOS client we need to add unit tests for reading/writing a plist file and for asynchronous network operations. I'm using SenTestingKit[1] as the unit testing framework. It's the default unit testing framework in ios sdk prior to ios 7. I'm mocking the network calls for unit testing asynchronous network operations. For that I'll be using OHHTTPStubs [2] framework which is under MIT license. If you have any ideas regarding this please let me know. [1]. http://www.quantum-step.com/download/sources/mystep/OCUnit/SourceCode/SenTestingKit/Documentation/IntroSenTestingKit.html [2]. https://github.com/AliSoftware/OHHTTPStubs Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Writing Unit tests for iOS EMM client
Hi Dilshan, That'll be very helpful for integration tests. Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Mon, May 5, 2014 at 3:31 PM, Dilshan Edirisuriya dils...@wso2.comwrote: Hi Harshan, For integrations tests I prefer you look at KIF at [1]. This is a widely used library. Its under Apache 2 license. Later on you can directly integrate this to a CI server as well. [1] - https://github.com/kif-framework/KIF [2] - https://github.com/kif-framework/KIF/blob/master/LICENSE Regards, Dilshan On Mon, May 5, 2014 at 2:02 PM, Harshan Liyanage hars...@wso2.com wrote: Hi, I have started $subject for WSO2 EMM next release. In the iOS client we need to add unit tests for reading/writing a plist file and for asynchronous network operations. I'm using SenTestingKit[1] as the unit testing framework. It's the default unit testing framework in ios sdk prior to ios 7. I'm mocking the network calls for unit testing asynchronous network operations. For that I'll be using OHHTTPStubs [2] framework which is under MIT license. If you have any ideas regarding this please let me know. [1]. http://www.quantum-step.com/download/sources/mystep/OCUnit/SourceCode/SenTestingKit/Documentation/IntroSenTestingKit.html [2]. https://github.com/AliSoftware/OHHTTPStubs Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. -- Dilshan Edirisuriya Senior Software Engineer - WSO2 Mob: + 94 777878905 http://wso2.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] Using maven to create build Android Cordova-Android projects in AppFactory
Hi, Currently I'm involved in using maven to create build Android / Cordova-Android application projects as a part of MEAP project which will be used to create build mobile application projects using AppFactory. There is a android-maven plugin [3] a maven-archetype [4] which can be used for building creating android applications. Using maven in android application projects has been explained well in [1]. However the maven plugin, archetypes android-maven dependencies in central maven repo are not provided by Google as mentioned in [4] [5]. These are provided by android-maven developer community. But they do not regularly update maven central repo with latest Android sdk dependencies [6]. Last dependency was released on August 2012 for Android 4.1.1.4 (sdk 16). Hopefully Google might provide android dependencies in future with Gradle based build system [9]. *Issue : *Lack of android dependencies in maven central repo *Solutions :* 1. Using maven-android deployer tool [7] to install android sdk dependencies to local m2 repo. This tool has been widely used by the developers who uses maven to build android projects. Then by installing these dependencies into wso2 nexus repo we could build Android maven projects. This solution will allow us to use above mentioned maven-android plugin archetype with slight modifications. 2. Using maven to invoke shell commands to create android project build it using Ant (default Android build system till Android switches to Gradle) I'm expecting to use first solution. Please provide your suggestions regarding this. *References* [1]. http://books.sonatype.com/mvnref-book/reference/android-dev.html [2]. http://stackoverflow.com/questions/5253029/why-arent-the-android-sdk-jars-in-any-maven-repository [3]. https://code.google.com/p/maven-android-plugin/ [4]. http://stand.spree.de/wiki_details_maven_archetypes [5]. https://groups.google.com/forum/#!topic/maven-android-developers/ZG9z6kQuPYo [6]. http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22com.google.android%22%20AND%20a%3A%22android%22 [7]. https://github.com/mosabua/maven-android-sdk-deployer [8]. http://developer.android.com/tools/building/index.html [9]. http://www.gradleware.com/android/gradle-the-new-android-build-system/ Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Using maven to create build Android Cordova-Android projects in AppFactory
Hi Ramith, Gradle for android is still a new technology for Android developers. AFAIK most of the android developers use default Ant based build for Android projects. I think going with maven /ant would be safer. Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Thu, Jul 17, 2014 at 11:12 AM, Ramith Jayasinghe ram...@wso2.com wrote: Hi Chathura, There could be problems since we don't have the support for gradle with in AF ( even if Jenkins supports it through a plug-in). Supporting Gradle might require some work ( and note that as far as I know we haven't thought of doing it now either). does majority of android developers use gradle ?. If its not the case I would propose going with what we have for now ( maven/ant) as the first cut and then figure out ways to improve on it. what do you guys think? regards Ramith On Wed, Jul 16, 2014 at 6:57 PM, Chathura Dilan chathu...@wso2.com wrote: Hi Harshan, Are there any difficulties to move to Gradle? I think they have all the dependencies including latest Android L preview On Wed, Jul 16, 2014 at 6:23 PM, Harshan Liyanage hars...@wso2.com wrote: Hi, Currently I'm involved in using maven to create build Android / Cordova-Android application projects as a part of MEAP project which will be used to create build mobile application projects using AppFactory. There is a android-maven plugin [3] a maven-archetype [4] which can be used for building creating android applications. Using maven in android application projects has been explained well in [1]. However the maven plugin, archetypes android-maven dependencies in central maven repo are not provided by Google as mentioned in [4] [5]. These are provided by android-maven developer community. But they do not regularly update maven central repo with latest Android sdk dependencies [6]. Last dependency was released on August 2012 for Android 4.1.1.4 (sdk 16). Hopefully Google might provide android dependencies in future with Gradle based build system [9]. *Issue : *Lack of android dependencies in maven central repo *Solutions :* 1. Using maven-android deployer tool [7] to install android sdk dependencies to local m2 repo. This tool has been widely used by the developers who uses maven to build android projects. Then by installing these dependencies into wso2 nexus repo we could build Android maven projects. This solution will allow us to use above mentioned maven-android plugin archetype with slight modifications. 2. Using maven to invoke shell commands to create android project build it using Ant (default Android build system till Android switches to Gradle) I'm expecting to use first solution. Please provide your suggestions regarding this. *References* [1]. http://books.sonatype.com/mvnref-book/reference/android-dev.html [2]. http://stackoverflow.com/questions/5253029/why-arent-the-android-sdk-jars-in-any-maven-repository [3]. https://code.google.com/p/maven-android-plugin/ [4]. http://stand.spree.de/wiki_details_maven_archetypes [5]. https://groups.google.com/forum/#!topic/maven-android-developers/ZG9z6kQuPYo [6]. http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22com.google.android%22%20AND%20a%3A%22android%22 [7]. https://github.com/mosabua/maven-android-sdk-deployer [8]. http://developer.android.com/tools/building/index.html [9]. http://www.gradleware.com/android/gradle-the-new-android-build-system/ Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. -- Regards, Chatura Dilan Perera *(Senior Software Engineer** - WSO2 Inc.* * [Mobile])* www.dilan.me -- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E: ram...@wso2.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] A mechanism to communicate server shutdown to the bundles, just before actual server shutdown
Hi Jayanga, Why can't register the CarbonServerShutdownPrepareService in normal OSGi service registration manner (without registering it in server shutdown)? Is that because you need to have synchronized manner or any other particular reason to do so? Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Mon, Sep 15, 2014 at 10:13 AM, Jayanga Dissanayake jaya...@wso2.com wrote: Hi Amila, Sorry about the confusion. Generally, in OSGi bundles, we register services provided by a particular bundle in the bundle activation, That is true. But in this scenario, I am registering the CarbonServerShutdownPrepareService in the org.wso2.carbon.core.init.CarbonServerManager.shutdownGracefully() method, which is called when the server about to shutdown. As actual server shutting down process is not yet started, all the bundles are active and the server is still in the full operational mode. So, we can register a new service without any harm. In OSGi framework, service registration, activation of dependent bundles, notifying listers and trackers happen in synchronized manner. So the shutdownGracefully() cannot proceed until the above service registration tasks completes. So, the bundles, which was waiting for the CarbonServerShutdownPrepareService can start doing its expected tasks (house keeping tasks that has to be carried out just before the server shutdown) Thanks Jayanga. *Jayanga Dissanayake* Senior Software Engineer WSO2 Inc. - http://wso2.com/ lean . enterprise . middleware email: jaya...@wso2.com mobile: +94772207259 On Sun, Sep 14, 2014 at 1:36 AM, Amila Maha Arachchi ami...@wso2.com wrote: On Thu, Sep 11, 2014 at 10:47 PM, Jayanga Dissanayake jaya...@wso2.com wrote: Hi, There was a requirement to detect the server shutdown from some bundles before the OSGi framework begin to shutdown. Because there were situations where some bundle have there own transports. If server shutdown happens while there were buffered messages, those should be processes before that particular bundle get deactivated. As the bundle stopping sequence is not guaranteed in OSGi environments, There may be some required bundles/services already being stopped when that particular bundle wants to do the finalizing tasks. So, as a solution I tried the following, Register an OSGi service CarbonServerShutdownPrepareService in the shutdownGracefully() before the actual server shutdown is called. So, just before the server shutdown CarbonServerShutdownPrepareService get activated. Then add some dummy OSGi Declarative Service components, that is waiting for ”CarbonServerShutdownPrepareService”, Then start the server and shutdown the server, Observations, When the server is shutting down, 1. first it registers the service “CarbonServerShutdownPrepareService” 2. then all the bundles waiting for that service get activated, (if there are listers or trackers, those get notified) 3. then actual server shutdown happens AFAIK, OSGi services are registered when the components startup. Here, you have said that CarbonServerShutdownPrepareService is registered when the server is shut down. This is not clear to me. Did you mean the following approach: 1. Register an OSGi service at the server startup. 2. Add declarative service dependencies to the bundles which needs to be notified about the shutdown. These bundles should implement a listener or a tracker. 3. When the server shutdown is called, it notifies the dependents. Is this sequential? It seems that I don't understand something here. If it is the case, please explain it to me. :) Thanks. As the osgi framework's service registration and un-registration happens in a single thread, It is guaranteed that registration of “CarbonServerShutdownPrepareService” and notification of that registration to the other bundles happens in a sequential manner in a single thread. So, any bundle can perform there house keeping tasks that has to carry out just before the server shutdown. Suggestions, improvements and alternatives are welcome Thanks, *Jayanga Dissanayake* Senior Software Engineer WSO2 Inc. - http://wso2.com/ lean . enterprise . middleware email: jaya...@wso2.com mobile: +94772207259 -- *Amila Maharachchi* Senior Technical Lead WSO2, Inc.; http://wso2.com Blog: http://maharachchi.blogspot.com Mobile: +94719371446 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Integrating BAM with EMM
Hi Inosh, Please find my comments inline. User's who have violated a policy during a certain period.(This will catch if the user violates a policy and then again adheres to it later) I think it would be better to get the list of users who have violated a particular policy first. Lets give the option to filter those results using the time interval. WDYT? How about getting the list of policies violated by an user or role? Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Mon, Sep 29, 2014 at 11:24 AM, Inosh Perera ino...@wso2.com wrote: Hi all, Following are some of the information we can extract once we integrate BAM, Please update the list if you see anything missing, Info- User's location during a certain time period How many times the user visited an area. Battery levels variation of a device during its life time or during a certain period(Can find out which devices performs well) Times where certain devices went offline. Apps- Installed App list of device Apps installed/uninstalled during a period Search for user's who have installed a certain app during any point of time Policy- User's who have cameras disabled/enabled. User's who have violated a policy during a certain period.(This will catch if the user violates a policy and then again adheres to it later) Policy violation frequency- per user/ over roll/ during a time period APIs- Track API usage during time period- such as, notifications, isregistered, monitoring Registrations- Registrations per users(register,unregister which we don't track now) Regards, Inosh On Wed, Sep 24, 2014 at 5:59 PM, Inosh Perera ino...@wso2.com wrote: Hi all, Based on an offline discussion we had with EMM team on the where to pick the data in order to publish, we decided it would be best, not to write a custom valve and handle it, instead collect all the data at EMM level. This is because, some data doesn't go through API manager valve currently. For example iOS bypasses the valve. Also based on a discussion with UES and BAM team, we decided to go with BAM dashboards to show statics, since these products are not yet integrated. Regards, Inosh On Thu, Sep 18, 2014 at 8:56 AM, Inosh Perera ino...@wso2.com wrote: Hi Dilshan, Stat publisher component is written in Java, and there will be APIs to publish each event to BAM. With current implementation of EMM, Jaggery will call these APIs when it is necessary to publish data. And this call will happen in a separate thread[1] and inside java publishing component also, we are using AsyncDataPublisher. Is this what you meant by asynchronously? Regarding which point should call to publish data; Issue in calling at API level is we have complex Json payloads coming from devices and also when we are responding. These things have to be parsed twice if we do at the API level. Also iOS doesn't go through API manager. We can do it in the driver.js where all the DB requests come, Is this a valid approach? True, doing this at function level will involves a lot of changes. [1] [DEV][Jaggery] Calling a Java method asynchronously from Jaggery Regards, Inosh On Thu, Sep 18, 2014 at 3:04 AM, Dilshan Edirisuriya dils...@wso2.com wrote: Hi Inosh, With the proposed refactoring discussions there is a high chance that most of the existing code get migrated to components. Hence is it possible to move this stat publisher logic into Java components and call them through API layer or some other generic point rather than adding them in actual functions. Also why not we focus on getting them done asynchronously where possible? Regards, Dilshan On Tue, Sep 16, 2014 at 9:00 PM, Inosh Perera ino...@wso2.com wrote: Hi Maninda/Gihan, Based on the offline discussions we had, your suggestions here seems to be the ideal solution for our scenario. Since the payloads coming to EMM have complex Json objects, writing a custom valve and extracting data doesn't seem like a good option. We will have to extract data at application level. In a situation like this, I can think of 2 places, where we can publish data when it comes to EMM. Lets say if you are inserting some record to the EMM database that we need to send to BAM as well, in this case, we can find the function that is sending the db request (for example register new user function) or there is a class in EMM that handles all the db request, something like a db object, which place seem like the ideal place to collect data? In my opinion, publishing data has to be done in each function. Regards, Inosh On Tue, Sep 16, 2014 at 4:53 PM, Maninda Edirisooriya mani...@wso2.com wrote: Hi Inosh, And you need to extract parameter fields in the Application level and note that these field set should be fixed for all the events.
Re: [Architecture] What is the best/wso2 way to authenticate REST endpoints.
Hi, Using OAuth will be beneficial future-proof as well. You can use it easily when the APIs are exposed to the public. +1 for using OAuth for API Security. Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Sun, Oct 19, 2014 at 7:29 PM, Manoj Gunawardena man...@wso2.com wrote: Hi, +1 for OAuth2. Because publisher APIs can be use in mobile devices. Ex -: customer implements mobile app to publish assets Also need to think about how customer can extend (customize) the security with our extension model. Ex-: Customer writes a extended publisher API and need to give different grant types and roles Also , I think better to maintain one security mechanism, rather than secure some apis with oAuth2 and some apis with Basic Authentication. Thanks On Sun, Oct 19, 2014 at 1:12 PM, Ayesha Dissanayaka aye...@wso2.com wrote: Thank you everyone for your valuable inputs. @Udara, These API endpoints are used by ES publisher App itself and will be invoked by authorized third party as well. In that way we have enabled accessing ES back office via remote clients as well. According to suggestions in this thread having aouth is the best way to secure the endpoints which are exposed to third party. We will decide whether to use basic-aouth/aouth or suppot both, and update the thread on final outcome. Thanks! - Ayesha On Sat, Oct 18, 2014 at 10:27 PM, Udara Liyanage ud...@wso2.com wrote: Hi, Having basic oauth with HTTPS is kind of secured as long as no third party is invoking the APIs. Touched, not typed. Erroneous words are a feature, not a typo. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Ayesha Dissanayaka* Software Engineer, WSO2, Inc : http://wso2.com http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg 20, Palmgrove Avenue, Colombo 3 E-Mail: aye...@wso2.com ayshsa...@gmail.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manoj Gunawardena Tech Lead WSO2, Inc.: http://wso2.com lean.enterprise.middleware Mobile : +94 77 2291643 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] Proposed Device Operations Flow of CDM
Hi, Please find the attached proposed device operations flow of CDM. Your suggestions feedback is highly appreciated. Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. Operation Flow (1).pdf Description: Adobe PDF document ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Proposed Device Operations Flow of CDM
Hi InoshP, Before explaining the above scenario I'll explain the process of payload generation when a request for new operation comes to the CDM core. 1. CDM Core bundle will detect its device platform using the operation code 2. Validate the operation against the supported device operations 3. Send the operation request to the device-platform bundle for converting it to a *device-specific payload* 4. Put the converted payload into the *OUT* queue We have two options when it comes to your scenario. 1. Create a new payload including the previous operation new operation 2. Create another payload. So that 2 payloads will be stored in the *OUT * queue. IMO the best option would be 1, since the second option will consume more memory in the queue more bandwidth of the device because it has to take 2 payloads. For supporting the option 1, we have the following options. 1. Device platform bundle will take the platform-specific payload in the queue merge it with the new operation In this case, the overhead of the device platform bundle will be much higher because it must know how to convert back the *platform-specific payload* to the *platform-independent* form. This will make writing a new device-platform bundle a tedious task because of the complexity of payload transformation logic. So I'm -1 on this option. 2. We'll use a platform-independent format for saving the payload into the queue which can be easily merged with new operations. This platform-independent payload will be converted when the device contacts the CDM server for pending operations. This is much more easier method than the first option. But this will affect the number of concurrent devices CDM can support due to the on-demand conversion of payloads. On the other-hand this method will reduce the resource consumption avoid unnecessary payload transformations. Further-more this will reduce the complexity of CDM core because there is no need to get the payload corresponding to the operation from the device-platform bundle put it into the OUT queue. 3. Save the platform-independent payload format along with the platform-specific payload in the queue. This method will increase the resource consumption because we need more memory to keep the platform-independent payload platform-dependent payload in the OUT queue. But this will increase the number of concurrent devices CDM can support because the payload is readily available when the device contacts the server. I'm +1 for both 2nd and 3rd options. WDYT? Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Mon, Nov 3, 2014 at 3:47 PM, Inosh Perera ino...@wso2.com wrote: Hi Harshan, If for example, a message to a device is already in the queue and when monitoring happen, a similar payload is generated. How is it handled when it comes to communication between out queue and device platform bundle? Regards, Inosh On Mon, Nov 3, 2014 at 3:20 PM, Harshan Liyanage hars...@wso2.com wrote: Hi, Please find the attached proposed device operations flow of CDM. Your suggestions feedback is highly appreciated. Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. -- Inosh Perera Software Engineer, WSO2 Inc. Tel: 0785293686 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Proposed Device Operations Flow of CDM
Hi Niranjan, Please find my comments in line. IMO this would be the a better option since the overhead is removed when the device tries to communicate with the server to get the payload. AFAIU even in the option 2 and 3, the device platform bundle should be able to convert the platform-independent payload to device specific payload. Therefore if in the platform-specific bundle has the reverse logic that is converting device specific to platform independent payload then the overhead is there at the time when a new operation is added. WDYT? Yes. But in 2, 3rd options there is no need to write the reverse logic to convert device-specific payload into platform-independent one. In option 3, the payload is readily available when device contacts the server. So sticking to the option 1 will increase the burden of the device platform bundle developers. I noticed that in the diagram, all the operations are put into the queue and the device contacts to retrieve the message. What about the scenario where the server would want to contact the device that is for example in Android and iOS, the server can connect the device via the GCM or APNS respectively. How is this handled? In the new architecture we have given the user to choose the preferred way of sending operations to the devices. Basically there will be 3 modes and I've only included the Local notification mode in this flow diagram. 1. Cloud based push notification mode (GCM, APNS) This is entirely based on cloud messaging services like GCM or APNS where the payload to be sent to the device is incorporated into a cloud message send to the device. 2. Local notification mode This method is similar to the existing Android Local notification system where the device periodically checks the server for pending operations. 3. Hybrid mode With this mode we'll use cloud messaging service to send a wake-up notification to the device. When the wake-up notification is received the device will contact the server for pending operations. So for supporting 1st 3rd mode, there must be an asynchronous background processor (as in *IN Queue*) which processes messages to be sent to the Cloud services. Device-platform bundle must have the logic to create such payloads send them to the cloud service. I'll update the diagram with these modes. Thanks for the reminding it. Thanks, Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Tue, Nov 4, 2014 at 10:14 AM, Niranjan Karunanandham niran...@wso2.com wrote: Hi Harshan 1. Device platform bundle will take the platform-specific payload in the queue merge it with the new operation In this case, the overhead of the device platform bundle will be much higher because it must know how to convert back the *platform-specific payload* to the *platform-independent* form. This will make writing a new device-platform bundle a tedious task because of the complexity of payload transformation logic. So I'm -1 on this option. IMO this would be the a better option since the overhead is removed when the device tries to communicate with the server to get the payload. AFAIU even in the option 2 and 3, the device platform bundle should be able to convert the platform-independent payload to device specific payload. Therefore if in the platform-specific bundle has the reverse logic that is converting device specific to platform independent payload then the overhead is there at the time when a new operation is added. WDYT? I noticed that in the diagram, all the operations are put into the queue and the device contacts to retrieve the message. What about the scenario where the server would want to contact the device that is for example in Android and iOS, the server can connect the device via the GCM or APNS respectively. How is this handled? Regards, Nira On Mon, Nov 3, 2014 at 5:36 PM, Harshan Liyanage hars...@wso2.com wrote: Hi InoshP, Before explaining the above scenario I'll explain the process of payload generation when a request for new operation comes to the CDM core. 1. CDM Core bundle will detect its device platform using the operation code 2. Validate the operation against the supported device operations 3. Send the operation request to the device-platform bundle for converting it to a *device-specific payload* 4. Put the converted payload into the *OUT* queue We have two options when it comes to your scenario. 1. Create a new payload including the previous operation new operation 2. Create another payload. So that 2 payloads will be stored in the *OUT *queue. IMO the best option would be 1, since the second option will consume more memory in the queue more bandwidth of the device because it has to take 2 payloads. For supporting the option 1, we have the following options. 1. Device platform bundle will take
Re: [Architecture] OSGI Service and Admin Services
Hi Godwin, IMO keeping the business logic separate from the services will be more appropriate. So that you can change the business logic later without breaking the service contract. Thanks, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Fri, Nov 14, 2014 at 11:29 AM, Godwin Amila Shrimal god...@wso2.com wrote: Hi All, Thanks for the feedback, I think I described it wrongly, I didn't mentioned 3 options to do this, I mentioned 3 steps to overcome the issue. Sorry for my bad. @Sameera : As you have mentioned we'll be have issues when admin services use axis2 context hierarchy e.g. MessageContext, properties set in Handlers etc, In that case what is the best way to implement, Do we have to keep separate business logic implementations for OSGIService and AdminService ? Thanks Godwin On Fri, Nov 14, 2014 at 11:22 AM, Sameera Jayasoma same...@wso2.com wrote: Hi Godwin, We tried the approach sometime back, but had to drop this coz of some complexities. * Some admin services needs to access the axis2 context hierarchy e.g. MessageContext, properties set in Handlers etc. So you cannot put all business logic to a OSGi Service. * Admin services can define permissions required to access them and the Security handlers checks whether the user is allowed access admin services. OSGi services which gives the same functionality as admin services allows components to use them freely without and security restrictions. AFAIR we faced such issues sometime back. This approach is okay for some OSGi services/admin services, but not for all. So we need to carefully think before exposing an Admin service as an OSGi service. Thanks, Sameera. On Thu, Nov 13, 2014 at 10:36 PM, Godwin Amila Shrimal god...@wso2.com wrote: Hi, We have Admin Services in carbon products which are needed to access as OSGI Services in the carbon context. Most of the time in Admin Services, it check weather user is authorized in the method, So we cannot register same Admin Service as a OSGI Service and access it. Since there are cases user is not authorized when we access as a OSGI Service. What is the best way to reuse the code and create OSGI Service and Admin Service. After discussing with some folks, i thought following mechanism to implement it. 1. Create an interface for the Service 2. Create an Implementation as OSGI Service 3. Create Admin Service which reuse the OSGI Service and apply additional security. (Here there will be no business logic and additional security will only applied.) Please give feedback on this. Thanks Godwin -- *Godwin Amila Shrimal* Senior Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware mobile: *+94772264165* linkedin: *http://lnkd.in/KUum6D http://lnkd.in/KUum6D* twitter: https://twitter.com/godwinamila ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Sameera Jayasoma, Software Architect, WSO2, Inc. (http://wso2.com) email: same...@wso2.com blog: http://sameera.adahas.org twitter: https://twitter.com/sameerajayasoma flickr: http://www.flickr.com/photos/sameera-jayasoma/collections Mobile: 0094776364456 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Godwin Amila Shrimal* Senior Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware mobile: *+94772264165* linkedin: *http://lnkd.in/KUum6D http://lnkd.in/KUum6D* twitter: https://twitter.com/godwinamila ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Proposal for Carbon Offsets
Hi, Yeah. I think having different offsets for different products will definitely confuse the customers who are already familiar with our products. But as Shammi has mentioned if we could have standard port-offsets for production deployments that might be useful in troubleshooting support issues. Thanks, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Thu, Nov 13, 2014 at 6:00 PM, Nuwan Dias nuw...@wso2.com wrote: Having different ports for different products will be a problem. Right now if one knows how to access the management console of a single product, he knows what to do to access any of the management consoles of any product in the platform. Similar to other ports as well, such as the ESB and APIM use 8280 as the default passthrough port, similar for default thrift ports as well. So having different offsets for different products will confuse users more IMO. On Thu, Nov 13, 2014 at 5:49 PM, Sameera Jayasoma same...@wso2.com wrote: I also have the same opinion on this change. All our users are familiar with the default http/https ports, changing them will confuse them. Thanks, Sameera. On Wed, Nov 12, 2014 at 8:33 PM, Afkham Azeez az...@wso2.com wrote: On Wed, Nov 12, 2014 at 6:12 PM, Isuru Perera isu...@wso2.com wrote: Hi, I'm -0 on this proposal. If we have different offsets for different products, we will have to maintain a document to show the offsets we have given for each product. Due to the same reason, I too think having multiple port offsets will be confusing since most users have got used to the standard ports, and are quite comfortable with setting different port offsets. When doing product integrations, setting offset is not the only configuration step. It is a simple operation and I think it's better to let the user to do that along with other configurations. Then the user will know what really happens with ports and how to configure URLs. In most of the production deployments, the WSO2 instances will be in separate servers/VMs and we rarely modify the offset. Most of the time, we define common security groups to open ports. So, if we don't change offsets by default, we will be having common ports for each server. So, I think it's better to have no offsets by default in WSO2 products. Thanks! Best Regards, On Wed, Nov 12, 2014 at 5:42 PM, Paul Fremantle p...@wso2.com wrote: Ack? Paul On 10 October 2014 09:27, Paul Fremantle p...@wso2.com wrote: I have a simple proposal, which is that we predefine a standard offset for each product. e.g. AS 0, ESB 1, AM 2, GovReg 3, BAM 4, IS 5, etc... These would be baked into the distros. This would make life a lot easier for customers doing more than one product. Also for say BAM integration, life would be much better. Paul -- Paul Fremantle CTO and Co-Founder, WSO2 OASIS WS-RX TC Co-chair, Apache Member UK: +44 207 096 0336 blog: http://pzf.fremantle.org twitter.com/pzfreo p...@wso2.com wso2.com Lean Enterprise Middleware Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, retransmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. -- Paul Fremantle CTO and Co-Founder, WSO2 OASIS WS-RX TC Co-chair, Apache Member UK: +44 207 096 0336 blog: http://pzf.fremantle.org twitter.com/pzfreo p...@wso2.com wso2.com Lean Enterprise Middleware Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, retransmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Isuru Perera Senior Software Engineer | WSO2, Inc. | http://wso2.com/ Lean . Enterprise . Middleware about.me/chrishantha -- *Afkham Azeez* Director of Architecture;
Re: [Architecture] Proposed Device Operations Flow of CDM
Hi, *Platform-specific payload *is the actual payload which will be delivered to the device when the device contacts the server for pending-operations. *Platform-independent *form is used to construct the *Platform-specific payload * communicate device operations internally within CDM. For example when an user sends a device operation using CDM web-console, *platform-independent *message will be sent from the console - CDM API - CDM Core - Device-platform bundle. Then the *device-platform bundle *will use it to construct the *platform-specific payload.* But there might be situation where some device operation payloads might not delivered to the device when another operation for that device comes-in. IMO in such cases its not good to have many pending *platform-specific payloads *for a single device. If we are to deliver multiple payloads that might introduce an additional overhead in network. In such situations what I suggest is to merge the new payload with the existing payload. To do that we have two options. 1. Use the existing *platform-specific payload * merge it with the new operation request 2. Use the *platform-independent *form merge it with new operation request (platform-independent form) construct a new payload Going through the 1st approach will be harder because then the device-platform bundle will have to have the all the conversion logic need to convert payloads ( *platform-independent - platform-specific * *platform-specific - platform-independent). *In some platforms converting back the *platform-specific payload* might not be possible because the device-platform bundle might use a 3rd party library to do the conversion. So we had to opt out 1st approach. If we follow the 2nd approach, CDM server itself can do the merging of the operation requests because it does not need any platform-specific knowledge to do so. Then the merged *platform-independent payload *can be send to the device-platform bundle to get the corresponding *platform-specific payload.* For do that we need to save the *platform-independent payload *along with the *platform-specific payload.* This approach will reduce the workload of device-platform implementer, which will make easier integration of new platforms. Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Mon, Nov 17, 2014 at 5:59 AM, Dilan Udara Ariyaratne dil...@wso2.com wrote: Hi All, This is just to clarify things. With reference to defining pending-operations-per-device, why do we need to have a *platform-specific payload *and its *platform-independent* *form*? What is the expected use-case of this? Regards, Dilan. *Dilan U. Ariyaratne* Software Engineer WSO2 Inc. http://wso2.com/ Mobile: +94775149066 lean . enterprise . middleware On Mon, Nov 3, 2014 at 5:36 PM, Harshan Liyanage hars...@wso2.com wrote: Hi InoshP, Before explaining the above scenario I'll explain the process of payload generation when a request for new operation comes to the CDM core. 1. CDM Core bundle will detect its device platform using the operation code 2. Validate the operation against the supported device operations 3. Send the operation request to the device-platform bundle for converting it to a *device-specific payload* 4. Put the converted payload into the *OUT* queue We have two options when it comes to your scenario. 1. Create a new payload including the previous operation new operation 2. Create another payload. So that 2 payloads will be stored in the *OUT *queue. IMO the best option would be 1, since the second option will consume more memory in the queue more bandwidth of the device because it has to take 2 payloads. For supporting the option 1, we have the following options. 1. Device platform bundle will take the platform-specific payload in the queue merge it with the new operation In this case, the overhead of the device platform bundle will be much higher because it must know how to convert back the *platform-specific payload* to the *platform-independent* form. This will make writing a new device-platform bundle a tedious task because of the complexity of payload transformation logic. So I'm -1 on this option. 2. We'll use a platform-independent format for saving the payload into the queue which can be easily merged with new operations. This platform-independent payload will be converted when the device contacts the CDM server for pending operations. This is much more easier method than the first option. But this will affect the number of concurrent devices CDM can support due to the on-demand conversion of payloads. On the other-hand this method will reduce the resource consumption avoid unnecessary payload transformations. Further-more this will reduce the complexity of CDM core because there is no need to get the payload
Re: [Architecture] Proposed Device Operations Flow of CDM
Hi Dilan, Yes. you are correct. Thanks, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Tue, Nov 18, 2014 at 7:20 AM, Dilan Udara Ariyaratne dil...@wso2.com wrote: Hi Harshan, This is with reference to the two merging options that you have mentioned in your previous reply to the thread. As I understood, with the 1st Option: You suggest to [1] convert platform-independent-payload to platform-specific-payload, save it and when another request comes in, [2] convert the platform-specific-payload back into its platform-independent-form and merge it with the platform-independent-payload for the new request and [3] convert that back to a platform-specific-payload. (Please correct me if I am wrong) So with this option, you suggest not to save a platform-independent-payload for the pending requests and deal with it only at run-time. As I understood, with the 2nd Option: You suggest to [1] Keep the platform-independent-payload saved along with its platform-specific-payload and when ever another request comes in, [2] Get the existing platform-independent-payload and merge it with new platform-independent-payload of the incoming request, save it and [3] Convert that to a new platform-specific-payload and overwrite the existing platform-specific-payload with the new one. (Please correct me if I am wrong) Thanks. *Dilan U. Ariyaratne* Software Engineer WSO2 Inc. http://wso2.com/ Mobile: +94775149066 lean . enterprise . middleware On Mon, Nov 17, 2014 at 8:16 AM, Harshan Liyanage hars...@wso2.com wrote: Hi, *Platform-specific payload *is the actual payload which will be delivered to the device when the device contacts the server for pending-operations. *Platform-independent *form is used to construct the *Platform-specific payload * communicate device operations internally within CDM. For example when an user sends a device operation using CDM web-console, *platform-independent *message will be sent from the console - CDM API - CDM Core - Device-platform bundle. Then the *device-platform bundle *will use it to construct the *platform-specific payload.* But there might be situation where some device operation payloads might not delivered to the device when another operation for that device comes-in. IMO in such cases its not good to have many pending *platform-specific payloads *for a single device. If we are to deliver multiple payloads that might introduce an additional overhead in network. In such situations what I suggest is to merge the new payload with the existing payload. To do that we have two options. 1. Use the existing *platform-specific payload * merge it with the new operation request 2. Use the *platform-independent *form merge it with new operation request (platform-independent form) construct a new payload Going through the 1st approach will be harder because then the device-platform bundle will have to have the all the conversion logic need to convert payloads ( *platform-independent - platform-specific * *platform-specific - platform-independent). *In some platforms converting back the *platform-specific payload* might not be possible because the device-platform bundle might use a 3rd party library to do the conversion. So we had to opt out 1st approach. If we follow the 2nd approach, CDM server itself can do the merging of the operation requests because it does not need any platform-specific knowledge to do so. Then the merged *platform-independent payload *can be send to the device-platform bundle to get the corresponding *platform-specific payload.* For do that we need to save the *platform-independent payload *along with the *platform-specific payload.* This approach will reduce the workload of device-platform implementer, which will make easier integration of new platforms. Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Mon, Nov 17, 2014 at 5:59 AM, Dilan Udara Ariyaratne dil...@wso2.com wrote: Hi All, This is just to clarify things. With reference to defining pending-operations-per-device, why do we need to have a *platform-specific payload *and its *platform-independent* *form*? What is the expected use-case of this? Regards, Dilan. *Dilan U. Ariyaratne* Software Engineer WSO2 Inc. http://wso2.com/ Mobile: +94775149066 lean . enterprise . middleware On Mon, Nov 3, 2014 at 5:36 PM, Harshan Liyanage hars...@wso2.com wrote: Hi InoshP, Before explaining the above scenario I'll explain the process of payload generation when a request for new operation comes to the CDM core. 1. CDM Core bundle will detect its device platform using
Re: [Architecture] [BAM] [Security] Securing REST API
+1 for using OAuth security for the APIs. Can't use guys use the API everywhere concept in this use-case? So that you can simply have the OAuth security for APIs. Best Regards, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Sun, Feb 1, 2015 at 7:30 AM, Maninda Edirisooriya mani...@wso2.com wrote: +1 for OAuth and IMO in future we should move all authorization in admin services to OAuth throughout the Carbon. It will be definitely possible when we are moving from SOAP to REST with Carbon 5. On Wed, Jan 28, 2015 at 6:03 PM, Prabath Siriwardena prab...@wso2.com wrote: +1 for using OAuth.. Please also think of the cost of maintaining and provisioning keys between servers in a clustered setup and the requirement of have an OAuth authorization server. Please see the approach suggested here [1] self-issued self-contained access tokens. This approach reduces all most all the overhead. [1]: http://blog.facilelogin.com/2014/10/self-issued-access-tokens.html Thanks regards, -Prabath On Wed, Jan 28, 2015 at 1:16 AM, Johann Nallathamby joh...@wso2.com wrote: On Tue, Jan 27, 2015 at 3:17 PM, Anjana Fernando anj...@wso2.com wrote: Hi, I guess our admin services are also accessible via basic auth, isn't it? .. We just thought, as a convenience method for the end user, they can use their username/password to access our API if required. So basically, if using OAuth, other than using SAML2 bearer token grant type or anything similar, is it possible to use the login username/password to our dashboard UI to generate the access token with resource owner credentials grant type maybe? .. This is also possible. But the access token has an finite expiry time. And it is not related to the browser session / not a moving window. So once it expires you must use the refresh token to get another access token. So this way user can login once and keep using APIs until they logout. Once they logout the access token can be revoked. Securing APIs with Basic Auth is also currently widely used. But it doesn't provide any advantage over OAuth2. So for future we should stick to OAuth2 only. For the validation of the OAuth2 token we should have a tomcat valve so that it can secure REST as well as SOAP services. I don't think we have written one all this time. Gihan if you are doing this can you sync up with IS team and lets finalize. Thanks, Johann. Cheers, Anjana. On Tue, Jan 27, 2015 at 2:42 PM, Supun Malinga sup...@wso2.com wrote: Hi Gihan, IMO using basic auth will make it vulnerable for dos attacks and less secure. So you need to think this thru. There is a possibility of authenticating already logged in users via the cookie data. But we will need to write a new cookie based oauth grant type for this. AFAIK we don't have such a grant type yet (Correct me if I'm wrong). On your latest note I think you can use the SAML2 grant type [0]. [0] https://docs.wso2.com/display/AM170/Token+API#TokenAPI-ExchangingSAML2bearertokenswithOAuth2(SAMLextensiongranttype) thanks, On Tue, Jan 27, 2015 at 1:48 PM, Gihan Anuruddha gi...@wso2.com wrote: No. We thought, it might convenient for the end user if we provide basic auth capabilities. We will integrate OAuth functionalities for our REST APIs. Regarding our requirement, We have multiple dashboards that validate the user through single login page. How can we do the backend API communication? Regards, Gihan On Tue, Jan 27, 2015 at 12:02 PM, Sumedha Rubasinghe sume...@wso2.com wrote: Any particular reason for securing product APIs using Basic Auth? Products like G-Reg, CDM are using OAuth 2.0 tokens for this instead. On Tue, Jan 27, 2015 at 11:53 AM, Gihan Anuruddha gi...@wso2.com wrote: Hi All, We are going to use a set of REST API [1] to communicate with the data layer. Basically, we are securing these REST APIs with basic auth. But we wanted to communicate with these REST APIs with already logged in user as well. Reason is we have a plan to use these REST API in our Message console dashboard and we want to have SSO kind of a logging solution for these dashboards without any individual login pages. So is it possible to use existing HTTP session cookie and authenticate REST API calls or do we have to use OAuth with some specific grant types? Appreciate your inputs here? [1] - [Architecture] BAM 3.0 REST APIs for AnalyticsDataService / Indexing / Search -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595 -- /sumedha m: +94 773017743 b : bit.ly/sumedha -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595 ___ Architecture mailing list Architecture@wso2.org
Re: [Architecture] BAM 3.0 REST APIs for AnalyticsDataService / Indexing / Search
Hi Gimantha,
AFAIK when writing REST APIs you have use POST verb to create a resource
and the resource data should be sent as the POST request body. I totally
agree with Sinthuja's comment regarding url pattern of *POST
/analytics/{tableName}*.
IMO you should not use POST methods to the following APIs as well and
instead you could use GET methods with query-strings. (since these methods
have been used to retrieve data)
6. Get the records from a table (By IDs)
*POST /analytics/records/{tableName}*
; tableName - The name of the table from which the records are retrieved.
; Content - A List of IDs of the records to be retrieved in the following
format.
[ id1 , id2 , id3 ]
; response - takes the format of the request content of No.7
14. Search records of a table
*POST /analytics/search*
; Content - takes the following format
{
tableName: sampleTableName,
language: sampleLanguageName,
query: sampleQuery,
start: start-location-of-the-result,
count: maximum-number-of-entries-to-return
}
Best Regards,
Lakshitha Harshan
Software Engineer
Mobile: *+94724423048*
Email: hars...@wso2.com
Blog : http://harshanliyanage.blogspot.com/
*WSO2, Inc. :** wso2.com http://wso2.com/*
lean.enterprise.middleware.
On Tue, Jan 20, 2015 at 1:29 AM, Gimantha Bandara giman...@wso2.com wrote:
Hi Sinduja,
Thank you for the feedback.
On Mon, Jan 19, 2015 at 12:04 PM, Sinthuja Ragendran sinth...@wso2.com
wrote:
Hi gimantha,
Please see the comments inline.
On Sun, Jan 18, 2015 at 11:24 PM, Gimantha Bandara giman...@wso2.com
wrote:
Hi,
Currently, I am working on $subject. Basically the methods in
AnalyticsDataService will be exposed through REST APIs. Please refer to
Architecture mail thread *[Architecture] BAM 3.0 Data Layer
Implementation / RDBMS / Distributed Indexing / Search* for more
Details. Following are the supported REST APIs.
1. Create a table
*POST /analytics/{tableName}*
; tableName - The name of the table to be created.
IMHO the above should be POST to '/analytics/*tables*' and the request
content should have the table name as given below.
{
tableName : Test
}
Since the POST takes only the table name, it is straightforward to use it
as a path parameter.
2. Delete a table
*DELETE /analytics/{tableName} *
; tableName - The name of the table to be deleted.
3. Check if a table exists
*GET /analytics/{tableName} *
; tableName - The name of the table being checked.
4. List All the tables
*GET /analytics/tables*
;Response will be an JSON array of table names. e.g. [ table1 ,
table2 , table3 ]
5. Get the records from a table.
*GET /analytics/records/{tableName}/{from}/{to}/{start}/{count} *
; tableName - The name of the table from which the records are retrieved.
; from - The starting time to get records from.
; to - The ending time to get records to.
; start - The paginated index from value
; count - The paginated records count to be read
; response - takes the format of the request content of No.7
Do we need to have 'records' in the URL? I think it's better to have
*/analytics/{tableName}/{from}/{to}/{start}/{count}
*
records is there to avoid conflicts with other contexts. As an example,
If we remove records, the URL will take the format
/analytics/{tableName}, which is already a defined REST API.
6. Get the records from a table (By IDs)
*POST /analytics/records/{tableName}*
; tableName - The name of the table from which the records are retrieved.
; Content - A List of IDs of the records to be retrieved in the
following format.
[ id1 , id2 , id3 ]
; response - takes the format of the request content of No.7
Similarly can we have this as * /analytics/{tableName}?*
7. Create records ( can be created in different tables or in the same )
*POST /analytics/records*
; Content - A list of records in json format like in below.
[
{
id: id1,
tenantId: -1234,
tableName: tableName1,
timestamp: -mm-dd hh:mm:ss,
values:
{
columnName1: value1,
columnName2: value2
}
},
{
id: id2,
tenantId: -1234,
tableName: tableName2,
timestamp: -mm-dd hh:mm:ss,
values:
{
columnName1: value1,
columnName2: value2
}
},
]
8. Delete records
*DELETE /analytics/records/{tableName}/{timeFrom}/{timeTo}*
; tableName - Name of the table from which the records are deleted.
; timeFrom - The starting time to delete records from.
; timeTo - The end time to delete records to.
Again do we need to have 'records' in the middle? IMHO
/analytics/{tableName}/{timeFrom}/{timeTo} is better.
9. Update records
*PUT /analytics/records*
; Content - As same as the POST method for creating records
10. Get the record count of table
*GET /analytics/count/{tableName}*
; tableName - The name of the table
11. Create Indices for a table
*POST
Re: [Architecture] BAM 3.0 REST APIs for AnalyticsDataService / Indexing / Search
Hi Gimantha,
Could you please explain the use-case for the API #6? IMO there should not
be any operation to fetch a list of records by ids. Instead there could be
an API which sends a list of records as the response for a query.
For the API #14 you can still use GET method with query strings. I think
you have included start count parameters to control the pagination. For
that you can use the HTTP range-header [1] or link headers as mentioned in
[2].
[1]. http://otac0n.com/blog/2012/11/21/range-header-i-choose-you.html
[2]. https://developer.github.com/v3/#pagination
Regards,
Lakshitha Harshan
Software Engineer
Mobile: *+94724423048*
Email: hars...@wso2.com
Blog : http://harshanliyanage.blogspot.com/
*WSO2, Inc. :** wso2.com http://wso2.com/*
lean.enterprise.middleware.
On Tue, Jan 20, 2015 at 6:34 PM, Gimantha Bandara giman...@wso2.com wrote:
Hi Harshan,
Thanks for the feedback. The list of IDs of the records to be retrieved
can be too long. So setting a list of ids as a query param is not
convenient. Even for the Search, we have to pass a query, which can be too
long. Thats why the post method is used for Search.
Thanks,
On Tue, Jan 20, 2015 at 12:52 PM, Sinthuja Ragendran sinth...@wso2.com
wrote:
Hi Gimantha,
I think following the conventions will be more intuitive to the users,
and we should have a proper reason to not follow. And the post request is
generally needs to be sent to create the object and the back end is going
to decide where to create the tables resource, therefore it should be POST
to '/analytics/tables' and message body should be having the table name. If
you want to use /analytics/{tableName}, then you should use PUT rather
POST [1]. But IMO POST is the most suitable operation for this context.
And through out the below given URL patterns, in order to fetch the
records from a table, you have used '/analytics/records/{tableName}' url
pattern where 'records' is in the middle and it is not the correct data
hierarchy and again I feel it's not the convention. Basically tables
contains records and not records contains tables. Therefore if we use
POST to '/analytics/tables' for create table, then you can use simply
user 'analytics/{tableName}' to GET/POST for the tables, IMHO the records
is just an additional segment which is not needed to be here.
[1] http://restcookbook.com/HTTP%20Methods/put-vs-post
Thanks,
Sinthuja.
On Tue, Jan 20, 2015 at 1:29 AM, Gimantha Bandara giman...@wso2.com
wrote:
Hi Sinduja,
Thank you for the feedback.
On Mon, Jan 19, 2015 at 12:04 PM, Sinthuja Ragendran sinth...@wso2.com
wrote:
Hi gimantha,
Please see the comments inline.
On Sun, Jan 18, 2015 at 11:24 PM, Gimantha Bandara giman...@wso2.com
wrote:
Hi,
Currently, I am working on $subject. Basically the methods in
AnalyticsDataService will be exposed through REST APIs. Please refer to
Architecture mail thread *[Architecture] BAM 3.0 Data Layer
Implementation / RDBMS / Distributed Indexing / Search* for more
Details. Following are the supported REST APIs.
1. Create a table
*POST /analytics/{tableName}*
; tableName - The name of the table to be created.
IMHO the above should be POST to '/analytics/*tables*' and the request
content should have the table name as given below.
{
tableName : Test
}
Since the POST takes only the table name, it is straightforward to use
it as a path parameter.
2. Delete a table
*DELETE /analytics/{tableName} *
; tableName - The name of the table to be deleted.
3. Check if a table exists
*GET /analytics/{tableName} *
; tableName - The name of the table being checked.
4. List All the tables
*GET /analytics/tables*
;Response will be an JSON array of table names. e.g. [ table1 ,
table2 , table3 ]
5. Get the records from a table.
*GET /analytics/records/{tableName}/{from}/{to}/{start}/{count} *
; tableName - The name of the table from which the records are
retrieved.
; from - The starting time to get records from.
; to - The ending time to get records to.
; start - The paginated index from value
; count - The paginated records count to be read
; response - takes the format of the request content of No.7
Do we need to have 'records' in the URL? I think it's better to have
*/analytics/{tableName}/{from}/{to}/{start}/{count}
*
records is there to avoid conflicts with other contexts. As an
example, If we remove records, the URL will take the format
/analytics/{tableName}, which is already a defined REST API.
6. Get the records from a table (By IDs)
*POST /analytics/records/{tableName}*
; tableName - The name of the table from which the records are
retrieved.
; Content - A List of IDs of the records to be retrieved in the
following format.
[ id1 , id2 , id3 ]
; response - takes the format of the request content of No.7
Similarly can we have this as * /analytics/{tableName}?*
7. Create records ( can be created in different tables or in the same )
*POST
Re: [Architecture] BAM 3.0 REST APIs for AnalyticsDataService / Indexing / Search
, it may not be that
informative what we are actually doing there.
[1]
https://www.google.com/search?q=REST+service+designgws_rd=ssl#q=REST+api+design
Cheers,
Anjana.
On Wed, Jan 21, 2015 at 3:35 AM, Gimantha Bandara giman...@wso2.com
wrote:
Hi,
Goood point!. Initially the search was implemented such a way that it
returns a list of ids of records that match the query. Now the search
method is changed, so it returns the records. So +1 for removing the API
#6.
@Sinduja, @Harshan
Thanks for the reference links and for the feedback.
On Tue, Jan 20, 2015 at 6:52 PM, Harshan Liyanage hars...@wso2.com
wrote:
Hi Gimantha,
Could you please explain the use-case for the API #6? IMO there
should not be any operation to fetch a list of records by ids. Instead
there could be an API which sends a list of records as the response for a
query.
For the API #14 you can still use GET method with query strings. I
think you have included start count parameters to control the
pagination.
For that you can use the HTTP range-header [1] or link headers as
mentioned
in [2].
[1]. http://otac0n.com/blog/2012/11/21/range-header-i-choose-you.html
[2]. https://developer.github.com/v3/#pagination
Regards,
Lakshitha Harshan
Software Engineer
Mobile: *+94724423048*
Email: hars...@wso2.com
Blog : http://harshanliyanage.blogspot.com/
*WSO2, Inc. :** wso2.com http://wso2.com/*
lean.enterprise.middleware.
On Tue, Jan 20, 2015 at 6:34 PM, Gimantha Bandara giman...@wso2.com
wrote:
Hi Harshan,
Thanks for the feedback. The list of IDs of the records to be
retrieved can be too long. So setting a list of ids as a query param is
not
convenient. Even for the Search, we have to pass a query, which can be
too
long. Thats why the post method is used for Search.
Thanks,
On Tue, Jan 20, 2015 at 12:52 PM, Sinthuja Ragendran
sinth...@wso2.com wrote:
Hi Gimantha,
I think following the conventions will be more intuitive to the
users, and we should have a proper reason to not follow. And the post
request is generally needs to be sent to create the object and the back
end
is going to decide where to create the tables resource, therefore it
should
be POST to '/analytics/tables' and message body should be having the
table
name. If you want to use /analytics/{tableName}, then you should
use PUT rather POST [1]. But IMO POST is the most suitable operation for
this context.
And through out the below given URL patterns, in order to fetch the
records from a table, you have used '/analytics/records/{tableName}' url
pattern where 'records' is in the middle and it is not the correct data
hierarchy and again I feel it's not the convention. Basically tables
contains records and not records contains tables. Therefore if we use
POST to '/analytics/tables' for create table, then you can use
simply user 'analytics/{tableName}' to GET/POST for the tables, IMHO the
records is just an additional segment which is not needed to be here.
[1] http://restcookbook.com/HTTP%20Methods/put-vs-post
Thanks,
Sinthuja.
On Tue, Jan 20, 2015 at 1:29 AM, Gimantha Bandara giman...@wso2.com
wrote:
Hi Sinduja,
Thank you for the feedback.
On Mon, Jan 19, 2015 at 12:04 PM, Sinthuja Ragendran
sinth...@wso2.com wrote:
Hi gimantha,
Please see the comments inline.
On Sun, Jan 18, 2015 at 11:24 PM, Gimantha Bandara
giman...@wso2.com wrote:
Hi,
Currently, I am working on $subject. Basically the methods in
AnalyticsDataService will be exposed through REST APIs. Please refer
to
Architecture mail thread *[Architecture] BAM 3.0 Data Layer
Implementation / RDBMS / Distributed Indexing / Search* for
more Details. Following are the supported REST APIs.
1. Create a table
*POST /analytics/{tableName}*
; tableName - The name of the table to be created.
IMHO the above should be POST to '/analytics/*tables*' and the
request content should have the table name as given below.
{
tableName : Test
}
Since the POST takes only the table name, it is straightforward to
use it as a path parameter.
2. Delete a table
*DELETE /analytics/{tableName} *
; tableName - The name of the table to be deleted.
3. Check if a table exists
*GET /analytics/{tableName} *
; tableName - The name of the table being checked.
4. List All the tables
*GET /analytics/tables*
;Response will be an JSON array of table names. e.g. [ table1 ,
table2 , table3 ]
5. Get the records from a table.
*GET /analytics/records/{tableName}/{from}/{to}/{start}/{count} *
; tableName - The name of the table from which the records are
retrieved.
; from - The starting time to get records from.
; to - The ending time to get records to.
; start - The paginated index from value
; count - The paginated records count to be read
; response - takes the format of the request content of No.7
Do we need to have 'records' in the URL? I think it's better to
have */analytics/{tableName}/{from}/{to}/{start}/{count
Re: [Architecture] BAM 3.0 REST APIs for AnalyticsDataService / Indexing / Search
Hi Gimantha,
I think Frank has explained everything about the usage of *GET
/analytics/tables/{tableName}*. Anyway what my point is that if we have
used *GET* */analytics/tables/{tableName} *to check the existence of the
table it would work nice if the table does not exist. But if the table
exists, all the table data will be retrieved from the database sent over
the network instead of a simple Boolean *TRUE* result. This will
introduce unnecessary performance drawback in your backend server
unnecessary network traffic.
So I'm +1 on using below API for that purpose.
*GET /analytics/table_exists?tableName=table-name*
Thanks,
Lakshitha Harshan
Software Engineer
Mobile: *+94724423048*
Email: hars...@wso2.com
Blog : http://harshanliyanage.blogspot.com/
*WSO2, Inc. :** wso2.com http://wso2.com/*
lean.enterprise.middleware.
On Mon, Feb 16, 2015 at 1:21 PM, Gimantha Bandara giman...@wso2.com wrote:
Hi Frank,
Thanks you for your explanation!. What we currently do is, using *GET
/analytics/tables/{tableName} *get the whole table, if the table does not
exist, HTTP status will be 404 and the response body will have a message
saying that the table does not exist.
On Sun, Feb 15, 2015 at 9:03 PM, Frank Leymann fr...@wso2.com wrote:
Hi Gimantha,
it depends on the scenario: if you want to check existence of resource
it's fine to use a GET on this resource and receive a 404 Not Found.
But the subtlety is that Not Found according to HTTP is statement in
time: you cannot infer that the resource does not exist, all that 404
says is that it cannot be found at this point in time, i.e. it maybe found
later. If your scenario doesn't care about that you are fine.
Furthermore, in case the resource in fact is found, the GET on this
resource will return the complete table. This might not be acceptable if
you only want to get an indicator that the exists. The signal for existence
shouldn't the possibly huge table itself.
Thus, depending on your scenario you may consider a corresponding
function. By the way, this is completely compliant to the REST style that
foresees such processing function resources.
Best regards,
Frank
2015-02-14 15:01 GMT+01:00 Gimantha Bandara giman...@wso2.com:
Hi Manuranga,
Already *GET /analytics/tables/{tableName} *returns 404 if the
table doesn't exists. So we will not need a separate API. Thanks for your
feedback.
On Sat, Feb 14, 2015 at 12:22 PM, Manuranga Perera m...@wso2.com
wrote:
there shouldn't be a separate end point for is-exists
*GET /analytics/tables/{tableName}* - Will return table informing if
it exists and if not it should return 404
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Gimantha Bandara
Software Engineer
WSO2. Inc : http://wso2.com
Mobile : +94714961919
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Gimantha Bandara
Software Engineer
WSO2. Inc : http://wso2.com
Mobile : +94714961919
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Pre-populate API Manager with one API which can be used OOTB
Hi Nisala, Normally it is unacceptable to have more than 3 arguments for a method when you write a clean code. I think you will have to refactor all these methods before proceed with your work because these methods already have too many arguments. As stated in [1], The ideal number of arguments for a function is zero (niladic). Next comes one (monadic), followed closely by two (dyadic). Three arguments (triadic) should be avoided where possible. More than three (polyadic) requires very special justification - and then shouldn't be used anyway. So please consider those things. [1]. http://www.javaworld.com/article/2074962/core-java/too-many-parameters-in-java-methods-part-8-tooling.html?null Thanks, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Tue, Feb 17, 2015 at 10:25 AM, Nisala Nanayakkara nis...@wso2.com wrote: I am working on the improvement of 'Pre-populating API Manager with one sample API in APIM'. The decided approach is create a simple API using inline implementation with script mediator in the server startup. For this,apim startup.publisher package will be re-used. There will be some changes in the startup.publisher package as below as currently that package doesn't support create APIs with inline implementation approach. Note-With below changes,the current functionality in startup publisher which is used by GReg/AS will not be break. - adding extra configuration property as 'isInline' in APIManager.xml to get the API implementation method.If this value is true,then use inline implementation approach,instead wrapping the API with a back-end endpoint. LocalAPIs LocalAPI Context/resource/Context Provideradmin/Provider Version1.0.0/Version IconPathnone/IconPath DocumentURLnone/DocumentURL AuthTypeAny/AuthType *isInlinetrue/isInline* /LocalAPI /LocalAPIs -changing createAPIModel and createAPIAtServerStartup method signatures as below private API createAPIModel(String apiName, String apiProvider, String apiVersion, String apiEndpoint, String apiContext, String iconPath, String documentURL, String authType, *boolean isInline*) private void createAPIAtServerStartup( String apiName, String apiProvider, String apiVersion, String apiEndpoint, String apiContext, String iconPath, String documentURL, String authType, *boolean isInline*) Please reply if you have any opinion about this. Thanks, Nisala -- *Best Regards,Nisala Niroshana Nanayakkara,* *Bsc. Eng Undergraduate | Department of Computer Science Engineering | University of Moratuwa | Sri Lanka* *Intern Software Engineer | WSO2 Lanka(pvt) Ltd * *Director | Leo Club of University of Moratuwa.* *Mobile | +94717600022* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] BAM 3.0 REST APIs for AnalyticsDataService / Indexing / Search
Hi Frank,
Got it. Thanks a lot.
Best Regards,
Lakshitha Harshan
Software Engineer
Mobile: *+94724423048*
Email: hars...@wso2.com
Blog : http://harshanliyanage.blogspot.com/
*WSO2, Inc. :** wso2.com http://wso2.com/*
lean.enterprise.middleware.
On Wed, Feb 18, 2015 at 5:58 PM, Frank Leymann fr...@wso2.com wrote:
Hi Harshan,
if you only want an indicator whether or not a table exists, the API you
specify is the right way to go :-)
And it's a nice RESTful API...
Best regards,
Frank
2015-02-17 6:08 GMT+01:00 Harshan Liyanage hars...@wso2.com:
Hi Gimantha,
I think Frank has explained everything about the usage of *GET
/analytics/tables/{tableName}*. Anyway what my point is that if we have
used *GET* */analytics/tables/{tableName} *to check the existence of the
table it would work nice if the table does not exist. But if the table
exists, all the table data will be retrieved from the database sent over
the network instead of a simple Boolean *TRUE* result. This will
introduce unnecessary performance drawback in your backend server
unnecessary network traffic.
So I'm +1 on using below API for that purpose.
*GET /analytics/table_exists?tableName=table-name*
Thanks,
Lakshitha Harshan
Software Engineer
Mobile: *+94724423048*
Email: hars...@wso2.com
Blog : http://harshanliyanage.blogspot.com/
*WSO2, Inc. :** wso2.com http://wso2.com/*
lean.enterprise.middleware.
On Mon, Feb 16, 2015 at 1:21 PM, Gimantha Bandara giman...@wso2.com
wrote:
Hi Frank,
Thanks you for your explanation!. What we currently do is, using *GET
/analytics/tables/{tableName} *get the whole table, if the table does
not exist, HTTP status will be 404 and the response body will have a
message saying that the table does not exist.
On Sun, Feb 15, 2015 at 9:03 PM, Frank Leymann fr...@wso2.com wrote:
Hi Gimantha,
it depends on the scenario: if you want to check existence of resource
it's fine to use a GET on this resource and receive a 404 Not Found.
But the subtlety is that Not Found according to HTTP is statement in
time: you cannot infer that the resource does not exist, all that 404
says is that it cannot be found at this point in time, i.e. it maybe found
later. If your scenario doesn't care about that you are fine.
Furthermore, in case the resource in fact is found, the GET on this
resource will return the complete table. This might not be acceptable if
you only want to get an indicator that the exists. The signal for existence
shouldn't the possibly huge table itself.
Thus, depending on your scenario you may consider a corresponding
function. By the way, this is completely compliant to the REST style that
foresees such processing function resources.
Best regards,
Frank
2015-02-14 15:01 GMT+01:00 Gimantha Bandara giman...@wso2.com:
Hi Manuranga,
Already *GET /analytics/tables/{tableName} *returns 404 if the
table doesn't exists. So we will not need a separate API. Thanks for your
feedback.
On Sat, Feb 14, 2015 at 12:22 PM, Manuranga Perera m...@wso2.com
wrote:
there shouldn't be a separate end point for is-exists
*GET /analytics/tables/{tableName}* - Will return table informing if
it exists and if not it should return 404
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Gimantha Bandara
Software Engineer
WSO2. Inc : http://wso2.com
Mobile : +94714961919
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Gimantha Bandara
Software Engineer
WSO2. Inc : http://wso2.com
Mobile : +94714961919
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] CDM Operation Managemen
Hi, Operation Management functionality will allow users, admins policy engines to remotely perform concrete operations on a device/devices. A Device Operation will contain the operation-code the payload of the operation (such as a configuration payload) if exists. An operation can be categorized into one of the following. - *Message* : CDM admins, users can send a message to a particular device. Typically these type of operations are one-way messages. - *Information* : CDM admins, users or policy engine (for evaluating the adherence to policies or as a requirement for dynamic policies) can request the current status of the device (location, battery level , temperature etc) - *Configuration* : CDM admins, users or policy engine can push a configuration such as WIFI configuration, camera disable , device mute, device lock to a particular device Following are the components involved in operation management. - *CDM UI : *CDM web console which will be used by the admins users - *Policy engine : *CDM Policy engine which will be used by admins to create policies to apply to the devices - *CDM Operation Manager : *This will be the core component which will handle all the operation management related tasks - *CDM DB : *CDM Core database - *Distributed Cache : *Distributed cache (Carbon cache) used to store retrieve operations - *Device APIs : * APIs which will be called by the devices - *Device : *An enrolled device (Can be any connected device such as a mobile device or an IOT device ) Operation Management can be mainly categorized into 3 areas. 1. *Add Operations* : Device operations can be added by the admins, users via CDM web-console or by the policy engine. In this scenario we need to persist the operation and applicable device list into the CDM database. Initial status of the operations in the database will be set to PENDING. Then we need to put these operations into the distributed-operation-cache for fast-retrieval. 2. *Fetch available Operations* : Devices will contact the CDM server to fetching available operations for those devices. This is a performance-critical function since it is the most-frequently used function in CDM. First CDM Operation Manager will do a cache-lookup for fetching the available operations. If that won't succeed it will do a database-lookup for available operations. Once the operation-manager receives the available operations for that particular device, it will remove those operations from the cache update the status of operations to SENT in database. *Issues* 1. How to exactly find whether there are any available operations for a particular device without doing a database query? Can't we assume that if there's no operation cache-entries for a particular device, there are no available operations for that device? 2. What will be the structure of the operation-cache? Is it a per-device cache or single cache for all devices? If it is a single cache what will be the key when putting a new operation to the cache? 3. Since this is a performance critical function is it advisable to update the operation status to SENT within this sequence? Can't we put the sent operations to another cache do the database update from another task? 3. *Update operation status*: After the received operations have been executed on the device, device will contact the server to update the status of the operation provide the information the operation has requested (i.e. location , temperature, status of camera etc). Once the update operation is contacted by the device, status of the appropriate operation will be changed to RECEIVED. *Issues* 1. Since a network call per each operation will drain the device battery how the device will update the status of multiple operations? Can't the agent application use a single network call to update the multiple operations? Given below is the proposed operation management sequence diagram of CDM. [image: Inline image 1] Please share your ideas suggestions upon this. Thanks, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Additional fields found in iOS mobile device management
Hi Dilshan, Currently the device table in mobile database has following fields. - MOBILE_DEVICE_ID - REG_ID - IMEI - IMSI - OS_VERSION - DEVICE_MODEL - VENDOR When considering iOS platform I think that we can map all other iOS fields to above fields except for the magic-token unlock-token. We can map the push-token to REG_ID field IMO I think that we can change the column name to *PUSH_TOKEN* since it is more generic term than *REG_ID* . For other two tokens we can use a separate field (if we are storing 2 tokens as a json string if those records are not frequently accessed) or 2 separate columns. In android-platform we have used WIFI Mac address as the MOBILE_DEVICE_ID. In your case I guess we'll need a separate column for storing mac address too. Since all other device information (device-capacity, bluetooth mac etc) will be available for all mobile platforms we'll need a generic way to handle these. I suggest to have a separate field in our device-table like *OTHER_INFO* and store all of these other as a json string as those information will be accessed as a whole. @Asok : Could you please share the list of fields available in Windows platform? Thanks, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Mon, Feb 16, 2015 at 11:42 AM, Dilshan Edirisuriya dils...@wso2.com wrote: Hi, Following are the additional fields found in iOS MDM other than the once we find in Android. Since our table structure is little bit tight to Android platform it is necessary to identify the extension points to store these fields as well along with Windows fields. 1) Challenge - since iOS get above information in 2 steps it has a challenge password embedded in the iOS payload. This is to verify the identify of the device together with its UDID. 2) Product - for this I will be sending this as either iPhone, iPad or iPod by looking at sub-strings. 3) Serial - serial number of the device. 4) Version - iOS device type. 5) IMEI - IMEI number. Incase there is no IMEI number it will return empty (ie iPad without the sim). 6) Model - model number of the device. 7) UDID - mapping to device identifier. 8) Token, Magic token and unlock token for normal push and MDM push notifications. After the enrollment using device information command we can fetch the device information such as AvailableDeviceCapacity, BluetoothMAC, BuildVersion, CarrierSettingsVersion, CurrentCarrierNetwork, CurrentMCC, CurrentMNC, DataRoamingEnabled, DeviceCapacity, DeviceName, ICCID, IMEI, IsRoaming, Model, ModelName, ModemFirmwareVersion, OSVersion, PhoneNumber, Product, ProductName, SIMCarrierNetwork, SIMMCC, SIMMNC, SerialNumber, UDID and WiFiMAC. Please note that these fields could vary based on the OS version of the device. We need to make these fields generic which will be helpful for the MDM to fetch data accurately in generic manner as well as store them generically so that any platform can fit into the existing architecture. Regards, Dilshan -- Dilshan Edirisuriya Senior Software Engineer - WSO2 Mob: + 94 777878905 http://wso2.com/ https://www.linkedin.com/profile/view?id=50486426 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] OSG level security
Hi, I also agree with Aruna's point. We have to trust the admin users who has physical access to the system. If those users are malicious users, they can even bring the entire system down if they want. In such cases I believe that we don't have anything to do. Thanks, Lakshitha Harshan Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com http://wso2.com/* lean.enterprise.middleware. On Sun, Feb 15, 2015 at 8:13 PM, Aruna Karunarathna ar...@wso2.com wrote: On Fri, Feb 13, 2015 at 9:39 PM, Godwin Amila Shrimal god...@wso2.com wrote: Hi, Since most of the hacking/fraud happens from the internally this topic just came to my mind, Our carbon products don't have OSGI level security, As an example, If someone internally in the company knows OSGI then can write an OSGI bundle which harm to the system and deploy simply. Shouldn't we consider this ? (Apologize if I am asking a question which is not valid) AFAIK Most Important Carbon API's are protected using Java Security, So the OSGi level security can be achieved using Java Security Manager. But from someone who has physical access to the system, we have to trust them. One thing we can do is, implement a separate server auditing mechanism (which is out of control from devops). Thanks Godwin -- *Godwin Amila Shrimal* Senior Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware mobile: *+94772264165* linkedin: *http://lnkd.in/KUum6D http://lnkd.in/KUum6D* twitter: https://twitter.com/godwinamila ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Aruna Sujith Karunarathna* | Software Engineer WSO2, Inc | lean. enterprise. middleware. #20, Palm Grove, Colombo 03, Sri Lanka Mobile: +94 71 9040362 | Work: +94 112145345 Email: ar...@wso2.com | Web: www.wso2.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Moving the OAuth 2.0 Dynamic Client Registration(DCR) feature to WSO2 Identity Server
Hi, Please find my comments inline. @Roshan, Yes that is the proper way of getting resource data according to Richardson maturity model for REST APIs.We provide resource ID and do get to fetch resource. I think we may add it for users to get data later. They can change grant types, scopes or app name by logging to management console. May be later we can support edit application with this web application. But as of now we focused on create and delete part only. But definitely we need to look at edit option as well. In that case we need to check user name and verify if app owner is edit app or not. Otherwise anyone can edit other peoples apps. We have done the same for EMM as well. However when a client invokes DCR endpoint (POST) for creating a oauth-client, we send the data of the oauth-client app if one already exists for the same client. I think we can do the same here. +1 for adding update API as well. But I think update API will be less likely to be used as there may be very less use-cases where we need to update the existing oauth-clients. So I believe you can add it later (In such extreme cases clients can be updated using Carbon console as Sanjeewa mentioned). Registration endpoint may named as oauth2/dcr or oauth2/register. If you use something like "oauth2/dcr" or "oauth2/register", you will run into problems for GET and PUT endpoints as the both resource definitions (dcr and register) does not make any sense when it comes to REST. I suggest you could use something like "oauth2/client" which is more informative and has a good resource definition. Thanks, Harshan Liyanage Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Fri, Feb 5, 2016 at 11:07 AM, Sanjeewa Malalgoda <sanje...@wso2.com> wrote: > As we discussed this endpoint will be secured with Basic Auth. > That means anyone who has login permission can create OAuth application in > system. > If need we may add special permission to create OAuth applications. > > There is a profile called open registration for OAuth applications but its > vulnerable to DoS attacks. > So we must use some form of authentication. > And spec says previously generated token also can use for this. But IMO it > will not suitable for our requirement. > > @Roshan, Yes that is the proper way of getting resource data according to > Richardson maturity model for REST APIs.We provide resource ID and do get > to fetch resource. > I think we may add it for users to get data later. > They can change grant types, scopes or app name by logging to management > console. May be later we can support edit application with this web > application. > But as of now we focused on create and delete part only. But definitely we > need to look at edit option as well. > In that case we need to check user name and verify if app owner is edit > app or not. Otherwise anyone can edit other peoples apps. > > Registration endpoint may named as oauth2/dcr or oauth2/register. > @Amila, having register is mandatory for registration URL? > If we let users to edit/delete as well then having resource name as > "register" may be bit of confusing. > > And most important thing this registration endpoint should secured with > TLS 1.2 as per spec section 5[ > https://tools.ietf.org/html/draft-ietf-oauth-dyn-reg-30#section-5]. > > Thanks, > sanjeewa. > > > > On Fri, Feb 5, 2016 at 12:34 AM, Amila De Silva <ami...@wso2.com> wrote: > >> Hi Tharika, >> >> Since *"**saasApp"* is not a common parameter as per the spec, you might >> have to emphasise the usage of it. >> >> Just noticed that in the excerpt you've provided request is sent to a >> resource named oauthdcr. Better if we can name it as /regsiter to show >> the conformity with the spec. >> >> On Thu, Feb 4, 2016 at 9:52 PM, Tharika Madurapperuma <thar...@wso2.com> >> wrote: >> >>> Hi All, >>> >>> We are planning of $subject. >>> >>> In some use cases, it is desirable or necessary to allow OAuth clients >>> to obtain authorization from an authorization server without the two >>> parties having previously interacted. Therefore in order for the >>> authorization server to accurately represent to end users which client is >>> seeking authorization to access the end user’s resources, a method for >>> automatic and unique registration of clients is needed. This is where the >>> Dynamic Client Registration protocol comes into play. >>> >>> The RFC related to this feature can be found in [1] >>> >>> *Problem* >>> >>&
Re: [Architecture] [EMM] Android agent auto enrollment
Hi, Is the service app signed by the vendor? if it so why don't we use the service app to get the serial number? Good point. I think that should be possible. If so we can skip steps 6 and 7. Thanks, Harshan Liyanage Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Thu, Jan 28, 2016 at 11:05 AM, Chathura Dilan <chathu...@wso2.com> wrote: > Hi Inosh, > > Could you explain why do you need connect it with the computer to retrieve > the serial number? > Is the service app signed by the vendor? if it so why don't we use the > service app to get the serial number? > > On Thu, Jan 28, 2016 at 10:34 AM, Harshan Liyanage <hars...@wso2.com> > wrote: > >> Hi Inosh, >> >> In the step 11, you have mentioned that the device sends authentication >> request, generate access and refresh tokens and send it to device. However >> you need client credentials (client key, secret) in-order to generate >> access tokens. How are you planing to get these client credentials prior to >> generating access tokens? In the existing EMM implementation we use >> Dynamic-client-registration to do that. I think we can use the same here. >> However we need to modify the flow diagram to reflect that. >> >> Thanks. >> >> Harshan Liyanage >> Software Engineer >> Mobile: *+94724423048* >> Email: hars...@wso2.com >> Blog : http://harshanliyanage.blogspot.com/ >> *WSO2, Inc. :** wso2.com <http://wso2.com/>* >> lean.enterprise.middleware. >> >> On Tue, Jan 26, 2016 at 2:07 PM, Inosh Perera <ino...@wso2.com> wrote: >> >>> Hi Milan, >>> >>> +1 Cant we use an embedded QR code reader or some other way to retrieve >>> this token? >>> Using QR code is possible and it would make the process usable in all >>> many platforms oppose to ADB where only Android is able to work with. >>> Although when it comes to COPE scenario, mostly it is Android devices and >>> it is a set of known devices so we can assume the driver change is not much >>> of a concern. So under the above assumption at the time of device getting >>> connected, we can think the drivers are already installed, which makes it >>> much easier to enroll because the user only needs to plugin the device. In >>> BYOP/CYOD scenario, this can be problematic. >>> >>> Regards, >>> Inosh >>> >>> >>> On Tue, Jan 26, 2016 at 12:29 PM, Milan Perera <mi...@wso2.com> wrote: >>> >>>> Hi Inosh, >>>> >>>> My concerns for the above proposed method as follows. >>>> >>>> AFAIU, in here what we are trying to do is to minimize the user >>>> interaction with the device as much as possible for the auto enrolment >>>> scenario. >>>> However according to above method, user should have to connect the >>>> device to a machine and has to run a script as well, hence it needs more >>>> interaction. >>>> Also, if we use ADB for this, there may be instance where PC does not >>>> recognize the device, which ends up manually installing drivers and etc. >>>> >>>> Why do we have to use ADB in order to do this? >>>> >>>> Cant we use an embedded QR code reader or some other way to retrieve >>>> this token? >>>> >>>> Regards, >>>> >>>> -- >>>> *Milan Perera *| Software Engineer >>>> WSO2, Inc | lean. enterprise. middleware. >>>> #20, Palm Grove, Colombo 03, Sri Lanka >>>> Mobile: +94 77 309 7088 | Work: +94 11 214 5345 >>>> Email: mi...@wso2.com <ar...@wso2.com> | Web: www.wso2.com >>>> <http://lk.linkedin.com/in/milanharinduperera> >>>> >>>> ___ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Inosh Perera >>> Software Engineer, WSO2 Inc. >>> Tel: 077813 7285, 0785293686 >>> >>> ___ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> ___ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Regards, > > Chatura Dilan Perera > *Senior Software Engineer** - WSO2 Inc.* > www.dilan.me > > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [EMM] Enabling application white listing and application blacklisting support
Hi Inosh, There may be some cases where enterprises need to have application policies for individual users. But I think that scenario is very unlikely. If we take an organization, every user will map to one or more user-roles. There might be situations where a role has only one user (i.e like CEO, MD). But still we can achieve it via the application policies for user-roles. Thanks, Harshan Liyanage Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Tue, Feb 2, 2016 at 9:37 AM, Inosh Perera <ino...@wso2.com> wrote: > Hi all, > > Role based application restriction will be provided. Administrator will > define a list of applications as a black list and a set of roles which is > to be restricted to the application, along with the applications. > Is there any particular reason for not having application policies for > individual users? > > Regards, > Inosh > > On Mon, Feb 1, 2016 at 11:05 PM, Prabath Abeysekera <praba...@wso2.com> > wrote: > >> >> On Mon, Feb 1, 2016 at 6:14 PM, Kasun Dananjaya Delgolla <kas...@wso2.com >> > wrote: >> >>> Hi Lakshman, >>> >>> In terms of Android you can use blocking APIs[1] in Marshmallow SDK (SDK >>> 23) to achieve this. We already use DevicePolicyManager API so you can >>> straightaway add these new stuff into the same android agent API layer. >>> Also for older API levels ( < 23) earlier we used a mechanism just to warn >>> the user if a blacklisted app is installed on the device since blocking of >>> apps is not supported in those API levels. >>> >> >> We might need to dig slightly deep into some of the APIs around and see >> if we've already got anything to mimic what's done in DevicePolicyManager, >> which is part of Marshmallow SDK; in previous versions of Android SDK. So, >> please check if there's any mechanism that'd potentially allow us to go >> beyond merely warning the user when a blacklisted application is installed >> and then block the installation completely particularly targeting SDKs < 23. >> >> Cheers, >> Prabath >> >> >>> >>> One more thing, we can add this to the system app which I'm in the >>> process of building. Then we can enable COPE (rooted/system access granted) >>> devices to blacklist/whitelist apps even though the API level is < 23. >>> >>> [1] - >>> http://developer.android.com/reference/android/app/admin/DevicePolicyManager.html >>> >>> Thanks >>> >>> On Mon, Feb 1, 2016 at 5:50 PM, Lakshman Udayakantha <lakshm...@wso2.com >>> > wrote: >>> >>>> Hi, >>>> >>>> There is a requirement to implement application white listing and >>>> application black listing support in Enterprise Mobility Manager. >>>> Application white listing means creating a list of applications which are >>>> only allowed to run on mobile devices which are connected to EMM. >>>> Application blacklisting is the opposite meaning in which there is a list >>>> of applications which are only not allowed to run on mobile devices which >>>> connected to EMM. >>>> As a solution for this we thought to introduce a configuration to >>>> identify black listing, white listing enabled or not and exactly which >>>> listing is enabled and If each configuration enabled separately EMM will >>>> behave in following manner. >>>> >>>> If ABL enabled, >>>> >>>> Role based application restriction will be provided. Administrator will >>>> define a list of applications as a black list and a set of roles which is >>>> to be restricted to the application, along with the applications. >>>> >>>> If AWL enabled, >>>> >>>> Administrator will check specific list of applications from admin UI. >>>> Only these applications will load on app store. Other means of applications >>>> installing will be blocked. >>>> 1. Blocking side-loading. >>>> 2. Third party app store blocking except EMM app store. >>>> 3. Google Play app blocking >>>> >>>> Any suggestions and thoughts are highly appreciated. >>>> >>>> Thanks >>>> -- >>>> Lakshman Udayakantha >>>> WSO2 Inc. www.wso2.com >>>> lean.enterprise.middleware >>>> Mobile: *0714388124 <0714388124>* >>>>
Re: [Architecture] [EMM] Android agent auto enrollment
Hi Inosh, In the step 11, you have mentioned that the device sends authentication request, generate access and refresh tokens and send it to device. However you need client credentials (client key, secret) in-order to generate access tokens. How are you planing to get these client credentials prior to generating access tokens? In the existing EMM implementation we use Dynamic-client-registration to do that. I think we can use the same here. However we need to modify the flow diagram to reflect that. Thanks. Harshan Liyanage Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Tue, Jan 26, 2016 at 2:07 PM, Inosh Perera <ino...@wso2.com> wrote: > Hi Milan, > > +1 Cant we use an embedded QR code reader or some other way to retrieve > this token? > Using QR code is possible and it would make the process usable in all many > platforms oppose to ADB where only Android is able to work with. Although > when it comes to COPE scenario, mostly it is Android devices and it is a > set of known devices so we can assume the driver change is not much of a > concern. So under the above assumption at the time of device getting > connected, we can think the drivers are already installed, which makes it > much easier to enroll because the user only needs to plugin the device. In > BYOP/CYOD scenario, this can be problematic. > > Regards, > Inosh > > > On Tue, Jan 26, 2016 at 12:29 PM, Milan Perera <mi...@wso2.com> wrote: > >> Hi Inosh, >> >> My concerns for the above proposed method as follows. >> >> AFAIU, in here what we are trying to do is to minimize the user >> interaction with the device as much as possible for the auto enrolment >> scenario. >> However according to above method, user should have to connect the device >> to a machine and has to run a script as well, hence it needs more >> interaction. >> Also, if we use ADB for this, there may be instance where PC does not >> recognize the device, which ends up manually installing drivers and etc. >> >> Why do we have to use ADB in order to do this? >> >> Cant we use an embedded QR code reader or some other way to retrieve this >> token? >> >> Regards, >> >> -- >> *Milan Perera *| Software Engineer >> WSO2, Inc | lean. enterprise. middleware. >> #20, Palm Grove, Colombo 03, Sri Lanka >> Mobile: +94 77 309 7088 | Work: +94 11 214 5345 >> Email: mi...@wso2.com <ar...@wso2.com> | Web: www.wso2.com >> <http://lk.linkedin.com/in/milanharinduperera> >> >> ___ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Inosh Perera > Software Engineer, WSO2 Inc. > Tel: 077813 7285, 0785293686 > > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Circuit Breaker Pattern for MSF4J
Hi Azeez, Does this timeout in open state occurs in exponentially (first timeout in 10 secs, next in 20 secs etc) or linearly when transitioning back to half-open state? For example if the state is in "Open" and now the timeout (lets say 10secs timeout) occurs. Then the state is moved to "half-open" state. But the next request is also a failure and breaker state is moved back to "open". In this occasion the what will be the timeout value? Is it 10 secs or 20 secs? Having an exponential timeout might be beneficiary here as it might save lot of resources if the service is continuously failing. But I think it would be better if we can provide both options in a configurable manner. So it is up to the developer to decide which method to use. Thanks, Harshan Liyanage Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Wed, Mar 30, 2016 at 5:05 AM, Afkham Azeez <az...@wso2.com> wrote: > I have written a sample which demonstrates circuit breaker in action; > http://blog.afkham.org/2016/03/microservices-circuit-breaker.html > > On Sat, Mar 12, 2016 at 6:09 PM, Afkham Azeez <az...@wso2.com> wrote: > >> This is a feature supported by some microservices frameworks. On the >> server side, in this case MSF4J runtime, failure counts are kept track of >> and then if the failures exceed certain thresholds, the circuit trips and >> rather than dispatch to the service, it returns service unavailable. >> >> Can you explain why this is not needed in a container environment? >> >> On Sat, Mar 12, 2016 at 12:56 PM, Sanjiva Weerawarana <sanj...@wso2.com> >> wrote: >> >>> I don't understand what server side circuit breaker means. How does the >>> server adjust itself? Where's that bit of logic running? >>> >>> IMO this is not needed in a container world. >>> >>> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez <az...@wso2.com> wrote: >>> >>>> Yes, that is client side circuit breaker. What Aruna is implementing is >>>> server side circuit breaker. Yes, we need both. >>>> >>>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <lak...@wso2.com> >>>> wrote: >>>> >>>>> Did you looked at [1] >>>>> >>>>> Netflix Hystrix <https://github.com/Netflix/Hystrix> is an incredibly >>>>> useful library for writing code that invokes remote services. Hystrix >>>>> times >>>>> out calls that exceed the specified threshold. It implements a *circuit >>>>> breaker* pattern, which stops the client from waiting needlessly for >>>>> an unresponsive service. If the error rate for a service exceeds a >>>>> specified threshold, Hystrix trips the circuit breaker and all requests >>>>> will fail immediately for a specified period of time. Hystrix lets you >>>>> define a fallback action when a request fails, such as reading from a >>>>> cache >>>>> or returning a default value. If you are using the JVM you should >>>>> definitely consider using Hystrix. >>>>> >>>>> [1] https://github.com/Netflix/Hystrix >>>>> >>>>> On Fri, Mar 11, 2016 at 2:42 PM, Aruna Karunarathna <ar...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi Devs, >>>>>> >>>>>> *Scenario* >>>>>> >>>>>> The deployed services in a MSF4J may fail to serve the requests due >>>>>> to various factors. e.g, >>>>>> 1. Less resources in the server. >>>>>> 2. High Load in the server >>>>>> 3, Some services take more time to respond etc. >>>>>> >>>>>> In this kind of a situation, if the server is getting requests though >>>>>> there is no resources to serve those requests, and eventually the server >>>>>> will get unusable. >>>>>> >>>>>> *Solution* >>>>>> >>>>>> The Circuit Breaker design pattern can save the server from above >>>>>> scenarios, The typical design can be illustrated as in the following >>>>>> diagram. >>>>>> >>>>>> >>>>>> So as in the above diagram, when number of failures of a particular >>>>>> resource exceeds the Max Failure Count, then the state of that resource >
Re: [Architecture] [CDMF] [EMM 2.1.0] Basic Use-cases for Dashboard Analytics Feature
Hi Dilan, Should n't we include some performance related dashboard as well? For example we can show no of connected devices per second, average response times, logged in user-count etc. I think it may not be feasible at this moment. But we can include such a dashboard in the next release. That'll be a nice feature to have. Thanks, Harshan Liyanage Software Engineer Mobile: *+94724423048* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Wed, Mar 30, 2016 at 7:08 AM, Dilan Udara Ariyaratne <dil...@wso2.com> wrote: > Hi Hasunie, > > Yes, Most of the use-cases that you have pointed out here are of-course > currently on consideration under > application related analytics. Not only "applications" related, but also > "user" and "policy" related use-cases are currently on review. > > I will update the thread ASAP on such requirements once they are sort of > finalyzed. > > Thanks, > Dilan. > > > > > > > *Dilan U. Ariyaratne* > Software Engineer > WSO2 Inc. <http://wso2.com/> > Mobile: +94766405580 <%2B94766405580> > lean . enterprise . middleware > > > On Tue, Mar 29, 2016 at 10:51 PM, Hasunie Adikari <hasu...@wso2.com> > wrote: > >> Hi Dilan, >> >> It should be add app related analytics like >> >> 1. Blacklisted app count >> 2. Recently updated app count >> 3. Recommended app count >> 4. Total device count not having latest app version. >> 5. App usage based on the platform, platform version >> 6. Most frequently used applications. >> 7. Application crash scenarios. >> >> >> On Tue, Mar 29, 2016 at 6:47 PM, Dilan Udara Ariyaratne <dil...@wso2.com> >> wrote: >> >>> Hi Geeth, >>> >>> Currently there is a plan to show device counts based on user defined >>> custom groups in EMM snapshot dashboard. >>> This is in addition to the platform based and ownership based default >>> grouping of devices. >>> >>> However, we may have to postpone plans on showing device counts based on >>> "INACTIVE", "BLOCKED" and "UNREACHABLE" states as >>> back-end support for maintaining such states are not yet available. >>> >>> Regards, >>> Dilan. >>> >>> >>> >>> >>> >>> *Dilan U. Ariyaratne* >>> Software Engineer >>> WSO2 Inc. <http://wso2.com/> >>> Mobile: +94766405580 <%2B94766405580> >>> lean . enterprise . middleware >>> >>> >>> On Fri, Feb 12, 2016 at 10:27 AM, Geeth Munasinghe <ge...@wso2.com> >>> wrote: >>> >>>> Hi Dilan, >>>> >>>> We may have to include the following too. >>>> >>>>1. Device count by groups, >>>>2. Device groups >>>>3. Active/Inactive devices >>>>4. Blocked/Removed devices >>>>5. Unreachable devices >>>>6. Number of enrolled devices within give time frame (new devices >>>>added) >>>> >>>> Thanks >>>> Geeth >>>> >>>> >>>> *G. K. S. Munasinghe* >>>> *Senior Software Engineer,* >>>> *WSO2, Inc. http://wso2.com <http://wso2.com/> * >>>> *lean.enterprise.middleware.* >>>> >>>> email: ge...@wso2.com >>>> phone:(+94) 777911226 >>>> >>>> On Fri, Feb 12, 2016 at 10:17 AM, Dilan Udara Ariyaratne < >>>> dil...@wso2.com> wrote: >>>> >>>>> Hi All, >>>>> >>>>> For the upcoming EMM 2.1.0 release, we have identified following basic >>>>> use-cases for >>>>> the Dashboard Analytics Feature with respect to devices. >>>>> >>>>> [1] Device count by Platform >>>>> [2] Device count by Ownership (BYOD, COPE) >>>>> [3] Device count by Security Concerns >>>>> - unmanaged (Meaning no policies currently assigned to the device) >>>>> - non-compliant (Meaning a policy has been already assigned, but >>>>> its rules have been maliciously overridden by some other third party) >>>>> - no passcode >>>>> - no encryption >>>>> [4] Last Seen Overview - Devices seen in last 8 hours >>>>> Last Seen Breakdown - Devices seen by dates (0-5, 5-15, 15-30, >>>>> >30) >>>>
Re: [Architecture] [IoT] Load Distribution for Device + Server Communication.
Hi all, Currently the bottleneck happens at getPendingOperations API. That method takes a while to be executed as it involves both fetching the next operations & updating the status of completed operations. The complete getPendingOperations flow has more than 5 database calls (as I remember) including a DB update. Hence most of the execution time would be spending on DB queries. A proper distribution of load might help us to safeguard server from sudden bursts for some extent. But IMO I think we should be also focusing on making getPendingOperations flow more efficient so that each device spends a lesser time in executing getPendingOperations API. For that we'll have to use queues for both ways (fetching / updating) so that we could fetch / update data from in-memory queue rather than from DB which is more efficient. Hence +1 for using Queues. Thanks, Harshan Liyanage EMM/IoT TG Mobile: *+94765672894* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Thu, Mar 2, 2017 at 12:10 PM, Kamidu Punchihewa <sachi...@wso2.com> wrote: > Hi All, > > Queuing the in and out communications seems to be the best option other > than handling the loading using times outs. But we need to come up with the > priority order when its come to the communication which is time sensitive > temperature monitoring in fire alarm etc. > > +1 for utilizing queues for communications going both ways. > > Thanks and Best Regards, > > Kamidu Sachith Punchihewa > *Software Engineer* > WSO2, Inc. > lean . enterprise . middleware > Mobile : +94 (0) 770566749 <%2B94%20%280%29%20773%20451194> > > Please Note that I have dyslexia and it may results in few misspelled > words in the content. > > Disclaimer: This communication may contain privileged or other > confidential information and is intended exclusively for the addressee/s. > If you are not the intended recipient/s, or believe that you may have > received this communication in error, please reply to the sender indicating > that fact and delete the copy you received and in addition, you should not > print, copy, retransmit, disseminate, or otherwise use the information > contained in this communication. Internet communications cannot be > guaranteed to be timely, secure, error or virus-free. The sender does not > accept liability for any errors or omissions. > > On Tue, Feb 28, 2017 at 7:41 AM, Charitha Goonetilleke <charit...@wso2.com > > wrote: > >> Hi All, >> >> I think it is better to have a solution which could be implemented in >> server side to distribute operations among devices. Because as we know >> device agents in IoTS devices are different to each other. In according to >> our architecture, we are letting anyone to write their own device types as >> well as the agents. Also we don't have our own agent for windows devices >> but using default windows workspace configurations to provide MDM. So >> implementing agent side solution would be more complex for some device >> types and will not move forward with all device types. >> >> Anyway as according what to suggested by Geeth & Menaka, we could use >> batch processing in server side to deliver operation to device batches >> rather than sending it to all devices. So we can use operation queue to >> deliver operations periodically to device batches. However we need to >> figure out few more things in there. >> >> >>1. How we figure out optimum number of devices we should call in a >>batch? >>2. What is the time span of delivering operation to all target >>devices? >>3. What we do if someone want to send operation immediately? >>4. Can we assure guaranteed delivery and can avoid repeated >>operations? >> >> For 1 & 2, we could use configuration to provide capability to set size >> of a batch and time span. However for 3, we need to have additional option >> to send operations immediately, probably using a priority queue. Also when >> it comes to 4, we need to keep records about the operations we have >> delivered and would be enough to add one additional operation status like >> "Queued". So we can change operation status to pending, when operation is >> delivered. >> >> WDYT? >> >> Thanks & Regards, >> /charithag >> >> On Tue, Feb 28, 2017 at 12:39 AM, Menaka Jayawardena <men...@wso2.com> >> wrote: >> >>> Hi, >>> >>> IMO we can send a timeout with the operation which, once the operation/ >>> notification is received by the device, it waits for the given tim
Re: [Architecture] Feature Management of IoT Server
Hi Ayyoob, That was the my concern as well. However we can consider the device-model instead of the device instance because ultimately all the supported features of a device will be dictated by the device model & vendor. By that way we can avoid lots of duplicate data. We can get this information from the device instance while it's enrolling as you have suggested. Then we can insert it to the tables only if that device model does not exists in the database. Thanks, Harshan Liyanage EMM/IoT TG WSO2 On Nov 2, 2016 10:03 PM, "Ayyoob Hamza" <ayy...@wso2.com> wrote: > I had an offline discussion and got an explanation from chathuraD. > > The problem here could be explained with a simple use case: Android has a > set of features but an instance of an android device doesn't have the > feature that is definied in the device type definition(eg: *Android OS > supports enabling/disabling fingerprint but the smart phone hardware > doesn't have a fingerprint scanner*). > > In our current implementation we don't support this. This is because we > have assumed that the device type has a set of feature and it is available > across all the instances. Therefore in the current approach when we send an > operation to a device and if the device doesn't support it then it sends a > feedback. This is not a synchronous flow and the user does not get a > realtime feedback whether the operation can be executed on the device or > not. > > However to solve this issue what we have todo is to maintain information > about the features a device(an instance) supports, This information can be > sent from the device when it enrolls/updates. In addition we need some form > of a compliance check to see whether the device has the particular feature > before sending this operation to the device. Implementing above requirement > will provide a good feed back mechanism to the user. > > @Chathura, In CDMF context we have defined that features is some thing > that we operate to. Sending read operation from server to device would be > just one use case but it doesn't need to be always, device can be > programmed to push data for a given interval or set a policy and trigger > with a conditional push. Therefore IMHO I think we need a separation. > Please correct me if I am wrong. > > Thanks, > Ayyoob > > *Ayyoob Hamza* > *Software Engineer* > WSO2 Inc.; http://wso2.com > email: ayy...@wso2.com cell: +94 77 1681010 <%2B94%2077%207779495> > > On Wed, Nov 2, 2016 at 3:44 PM, Chathura Ekanayake <chath...@wso2.com> > wrote: > >> Yes.. in the feature definition xml I sketched out, there are multiple >> criteria under tag. >> >> On Wed, Nov 2, 2016 at 2:55 PM, Chathura Dilan <chathu...@wso2.com> >> wrote: >> >>> Hi Chathura, >>> >>> How about having multiple criteria under one feature so we can better >>> address all the scenarios. >>> >>> >>> On Wed, Nov 2, 2016 at 12:36 PM, Chathura Ekanayake <chath...@wso2.com> >>> wrote: >>> >>>> Regarding sensors, we can consider them as features, where the action >>>> they support is "read value". If we are using polling method to trigger >>>> operations, read operation can have a parameter with an endpoint to send >>>> sensor reading. >>>> >>>> - Chathura >>>> >>>> On Wed, Nov 2, 2016 at 12:30 PM, Chathura Ekanayake <chath...@wso2.com> >>>> wrote: >>>> >>>>> I think we can define a feature (or an operation) with: >>>>> >>>>> A) Feature code - to identify the feature >>>>> B) Filtering criteria - to identify devices which can perform the >>>>> operation >>>>> C) Operation parameters - Additional details needed by the operation >>>>> >>>>> Restriction criteria such as device type, group ID, device version >>>>> (better a range of versions) can come under B. So whenever an operation is >>>>> invoked, IOT platform should evaluate device details against B and decide >>>>> whether to permit the operation. If permitted, it can get C from the user >>>>> (or from an external system) and add an operation record in DB with A and >>>>> C >>>>> against the device ID. >>>>> >>>>> With this we can use feature schema similar to what Ruwan has proposed: >>>>> >>>>> >>>>> >>>>> abc >>>>> this is a feature >>>>> >>>>> DT1 >>
Re: [Architecture] Feature Management of IoT Server
Hi Ayyoob, Yes you are correct. But I took the barometer as an example. However there may be real scenarios where some devices does not have GPS so that we can't access the location info. In such cases we need to disable "Location" operation for those devices. Thanks, Harshan Liyanage EMM/IoT TG Mobile: *+94765672894* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Wed, Nov 2, 2016 at 10:07 AM, Ayyoob Hamza <ayy...@wso2.com> wrote: > @Harshan, sensors are not part of features(correct me if I am wrong), This > is why there was a requirement raised for sensor management as part of > device management and we had an implementation to support this(from shabir) > which is not merged yet. > > [1] https://github.com/wso2/carbon-device-mgt/pull/407 > <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fwso2%2Fcarbon-device-mgt%2Fpull%2F407=D=1=AFQjCNGLqXZzNEVnTBLB5rugQeJWfFmnNQ> > [2] https://github.com/wso2/carbon-device-mgt-plugins/pull/394 > <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fwso2%2Fcarbon-device-mgt-plugins%2Fpull%2F394=D=1=AFQjCNFHWDeJJW5WCMCeENFnDo6271makQ> > > > *Ayyoob Hamza* > *Software Engineer* > WSO2 Inc.; http://wso2.com > email: ayy...@wso2.com cell: +94 77 1681010 <%2B94%2077%207779495> > > On Wed, Nov 2, 2016 at 9:54 AM, Harshan Liyanage <hars...@wso2.com> wrote: > >> Hi Dilan, >> >> +1 for the idea. But we also need to consider the device model when comes >> to the features. For example there may be Android devices where some >> sensors like barometer is not present. This will be mostly applicable to >> the mobile device types because IOT devices will have their own plugin >> independent of the device model (correct me if I'm wrong). >> >> Thanks, >> >> Harshan Liyanage >> EMM/IoT TG >> Mobile: *+94765672894* >> Email: hars...@wso2.com >> Blog : http://harshanliyanage.blogspot.com/ >> *WSO2, Inc. :** wso2.com <http://wso2.com/>* >> lean.enterprise.middleware. >> >> On Wed, Nov 2, 2016 at 9:14 AM, Jasintha Dasanayake <jasin...@wso2.com> >> wrote: >> >>> Hi All >>> >>> Yes.. , engaging with a plugin repository like eclipse vorto would be >>> great idea for IOT , we can provide set of extension point (we may already >>> have) where custom plugins can be plugged in >>> >>> >>> Thanks >>> /Jasintha >>> >>> On Wed, Nov 2, 2016 at 7:53 AM, Ruwan Yatawara <ruw...@wso2.com> wrote: >>> >>>> >>>> On Wed, Nov 2, 2016 at 7:23 AM, Ayyoob Hamza <ayy...@wso2.com> wrote: >>>> >>>>> different >>>> >>>> >>>> I am with Ayyoob in that the term "feature" sounds very much something >>>> from the mobile world. But if you think about it, any functionality that a >>>> user sees on the dashboard of a device type is a feature. So I am +1 for >>>> bringing feature definition down to the mobile plugin in form like. >>>> >>>> >>>> >>>> abc >>>> 5.0.0 >>>> this is a feature >>>> >>>> place_holder >>>> place_holder >>>> place_holder >>>> >>>> >>>> >>>> >>>> But somewhere down the line when it comes to orchestrating device >>>> integration flows, we most definitely as Ayyoob mentioned, have to take a >>>> look at Vorto... which is very cool IMO. >>>> >>>> >>>> Thanks and Regards, >>>> >>>> Ruwan Yatawara >>>> >>>> Associate Technical Lead, >>>> WSO2 Inc. >>>> >>>> email : ruw...@wso2.com >>>> mobile : +94 77 9110413 >>>> blog : http://ruwansrants.blogspot.com/ >>>> https://500px.com/ruwan_ace >>>> www: :http://wso2.com >>>> >>>> >>> >>> >>> -- >>> >>> *Jasintha Dasanayake**Associate Technical Lead* >>> >>> *WSO2 Inc. | http://wso2.com <http://wso2.com/>lean . enterprise . >>> middleware* >>> >>> >>> *mobile :- 0711-368-118* >>> >> >> > ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Feature Management of IoT Server
Hi Dilan, +1 for the idea. But we also need to consider the device model when comes to the features. For example there may be Android devices where some sensors like barometer is not present. This will be mostly applicable to the mobile device types because IOT devices will have their own plugin independent of the device model (correct me if I'm wrong). Thanks, Harshan Liyanage EMM/IoT TG Mobile: *+94765672894* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Wed, Nov 2, 2016 at 9:14 AM, Jasintha Dasanayake <jasin...@wso2.com> wrote: > Hi All > > Yes.. , engaging with a plugin repository like eclipse vorto would be > great idea for IOT , we can provide set of extension point (we may already > have) where custom plugins can be plugged in > > > Thanks > /Jasintha > > On Wed, Nov 2, 2016 at 7:53 AM, Ruwan Yatawara <ruw...@wso2.com> wrote: > >> >> On Wed, Nov 2, 2016 at 7:23 AM, Ayyoob Hamza <ayy...@wso2.com> wrote: >> >>> different >> >> >> I am with Ayyoob in that the term "feature" sounds very much something >> from the mobile world. But if you think about it, any functionality that a >> user sees on the dashboard of a device type is a feature. So I am +1 for >> bringing feature definition down to the mobile plugin in form like. >> >> >> >> abc >> 5.0.0 >> this is a feature >> >> place_holder >> place_holder >> place_holder >> >> >> >> >> But somewhere down the line when it comes to orchestrating device >> integration flows, we most definitely as Ayyoob mentioned, have to take a >> look at Vorto... which is very cool IMO. >> >> >> Thanks and Regards, >> >> Ruwan Yatawara >> >> Associate Technical Lead, >> WSO2 Inc. >> >> email : ruw...@wso2.com >> mobile : +94 77 9110413 >> blog : http://ruwansrants.blogspot.com/ >> https://500px.com/ruwan_ace >> www: :http://wso2.com >> >> > > > -- > > *Jasintha Dasanayake**Associate Technical Lead* > > *WSO2 Inc. | http://wso2.com <http://wso2.com/>lean . enterprise . > middleware* > > > *mobile :- 0711-368-118* > ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] [Dev] [VOTE] Release WSO2 Enterprise Mobility Manager 2.2.0 RC2
Hi Devs, This is the release candidate of WSO2 Enterprise Mobility Manager 2.2.0. Please download EMM 2.2.0 RC2 and test the functionality and vote. Vote will be open for 72 hours or as needed. Know issues: https://wso2.org/jira/issues/?filter=13384 Fixes provided : https://wso2.org/jira/issues/?filter=13582 <https://wso2.org/jira/issues/?filter=13582> Source & binary distribution files: https://github.com/wso2/product-emm/releases/tag/v2.2.0-RC2 The tag to be voted upon: https://github.com/wso2/product-emm/tree/release-2.2.0-RC2 [+] Stable - go ahead and release [-] Broken - do not release (explain why) Thanks and Regards, Harshan Liyanage EMM/IoT TG Mobile: *+94765672894* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] FCM for Android
Hi Pasindu, It was the mechanism we have used in EMM 1.1.0 with GCM. But AFAIK the intention of GCM or FCM or anyother notification service (i.e APNS) is to just send the wakeup call to the device. That's the reason behind we have changed our implementation (from EMM 2.0.0 onwards) to send only the wakeup call. Thanks, Harshan Liyanage EMM/IoT TG WSO2 On Dec 1, 2016 9:54 AM, "Pasindu Upulwan" <pasindujw...@cse.mrt.ac.lk> wrote: > Hi Inosh, > > Instead of letting the app to fetch the operation from the IoT/EMM server > (step 7 in the diagram), is it possible to attach the operation information > also in the wakeup message itself? The FCM size limit of 4096 bytes will be > enough to attach most of the basic operation payloads such as ring > operation, mute operation, etc. That will reduce the amount of > communications the app needs to do with the sever to execute an operation. > Perhaps after doing the operation, the app may send a 'delivery report' to > the IoT/EMM server if the delivery report of FCM is not enough to prove the > completion of the operation sent to the app. > > Thanks. > > On 30 November 2016 at 22:58, Inosh Perera <ino...@wso2.com> wrote: > > > > > > Hi Dilan, > > > > [1] Are we bringing FCM as a replacement for GCM or as another Push > Notification Mechanism that can be enabled apart from GCM ? > > >> > > >> FCM is a replacement for GCM. With deprecation of GCM(1), we have moved > to FCM. > > > > > > [2] If as a replacement, what advantages are there with FCM over GCM ? > > >> > > >> Firebase platform has more features compared to GCM(2) which we can be > utilised in the future for example crash reporting. > > > >> > >> FCM has an improved message delivery speed which is said to be 95% of > total messages be delivered within 250ms(3) > >> Firebase is cross platform(4) > > > >> > >> > > > > [3] When it comes to Google-services.json, It seems that it is required > to obtain a separate Google-services.json + FCM server token per each > production deployment and > > compile the agent embedding this file. If that's the case here, > cannot we achieve the same without letting a user to compile the source ? > > Each customer need to compile the agent and have this file > available in the root directory as mandated by Google(5). If this is not > available, Android studio will not allow you to compile the agent/any other > app that uses FCM. > > > > what if the device could retrieve this file at the time of enrollment > with the server and we provide this configuration to the server via > platform configurations ? > > > >> > >> Since this is in source folder, the original json file need to be > available at compile time and we cannot replace it on-the-fly at runtime. > > > > > > [4] And at last a general question, architecturally, how FCM differs > from GCM ? > > As mentioned main differences is in client side. Earlier we defined > a key called the sender ID in the platform configurations and this was > delivered to the device after authentication. With this ID, we initiated > registration with GCM server. However with FCM, there is no sender ID, but > all the configurations are stored in a json file we discussed earlier and > at the time of app's first initialisation, token retrieval will happen > automatically. This token is now called FCM instance ID token which was > earlier known as the GCM registration ID. Google has also replaced the > earlier "API key" which was used to invoke GCM server with "firebase cloud > messaging token" to establish the link with FCM server. FCM instance ID > token is provided instead of GCM registration ID when communicating with > FCM server. > > > > (1). https://developers.google.com/cloud-messaging/android/legacy-regid# > registration-state-sync-handling > > (2). https://firebase.google.com/features/ > > (3). https://www.youtube.com/watch?v=sioEY4tWmLI > > (4). https://firebase.google.com/docs/cloud-messaging/ > > (5). https://firebase.google.com/docs/android/setup > > > > Regards, > > Inosh > > > > On Wed, Nov 30, 2016 at 9:25 PM, Dilan Udara Ariyaratne <dil...@wso2.com> > wrote: > > >> > > >> Hi Inosh and All, > >> > >> Having following questions regarding this feature and the enabling > technology : > >> > >> [1] Are we bringing FCM as a replacement for GCM or as another Push > Notification Mechanism that can be enabled apart from GCM ? > >> [2] If as a replacement, what advantages are there wit
Re: [Architecture] [IoT] Simplifying IoT device type plugin with a descriptor
Hi Ayyoob, There are some instances where we need to cater plugin-specific functionalities which can't be achieved through a boiler-plate implementation. This behavior can be seen in mostly mobile-device plugins. For most of the times we don't need specific internal implementation. For example sometimes we need to store some device data in plugin database table and fetch it using a specific query. How are we planning to move forward in such scenarios? Can this deployer generate tables other than device table and feature table? If not I suggest to add following capabilities to the deployer model. - Ability to specify custom tables - Ability to specify custom queries/DAOs - Extension model where we can specify custom implementations which are required for the device plugin WDYT? Thanks, Harshan Liyanage EMM/IoT TG Mobile: *+94765672894* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Tue, Nov 22, 2016 at 10:10 PM, Ayyoob Hamza <ayy...@wso2.com> wrote: > Hi Dilan, > > >> Does this mean we can basically have a ready-to-use device plug-in >> implementation, just by configuring a descriptor file ? >> > Yes thats the idea, In here the plugin means the osgiService which > contains the definition of the device type. we found from the existing > device type that it all follows the same implementation template as the > plug-in(DeviceManagementService) implementation. Therefore I created a > generic template which populates the DeviceManagementService through a > configuration file. > > Thanks, > Ayyoob > > > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Restructuring Device Type Plugins for IoT
Hi Ruwan, We have included the notification providers like GCM, MQTT, FireBase inside the core (in device-mgt-extensions component) in the existing implementation of device-mgt. I do agree with your comment that says those are only extensions to the core framework. Have we thought of moving them out from core as a plugin-extension? Thanks, Harshan Liyanage EMM/IoT TG Mobile: *+94765672894* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Wed, Nov 23, 2016 at 9:26 AM, Ruwan Yatawara <ruw...@wso2.com> wrote: > @Dilan, > The idea here is that things like the transports adapters and the > appm-connector are modular plugins that are just complementing the CDMF > which is s framework. The framework does not necessarily need these to > function and perform its core tasks. Hence these should evolve/get added > independently from the framework. Furthermore given that, once the core > matures, releases of these extensions will be more frequent than changes to > the core, hence all the more need for this to be managed separately. We may > later even need to make a separate repository out of these extensions, as > one can also argue that such extensions do not belong in the device-type > repo. > > @Chathura, > As of this moment, we do not promote dynamic deployment of device types. > But yes, you are right we will have to think about this at some point down > the line when we support same. > > Thanks and Regards, > > Ruwan Yatawara > > Associate Technical Lead, > WSO2 Inc. > > email : ruw...@wso2.com > mobile : +94 77 9110413 > blog : http://ruwansrants.blogspot.com/ > https://500px.com/ruwan_ace > www: :http://wso2.com > > > On Wed, Nov 23, 2016 at 5:47 AM, Chathura Dilan <chathu...@wso2.com> > wrote: > >> Hi Ruwan, >> >> How about having a UI for the plugin deployment? So plugins can be >> installed and uninstalled via the UI like how we install/uninstall features >> in the carbon console. >> >> On Tue, Nov 22, 2016 at 9:59 PM, Dilan Udara Ariyaratne <dil...@wso2.com> >> wrote: >> >>> Hi Ruwan, >>> >>> Why would not analytics be inside a particular device type ?, this is in >>> fact the idea provided in [1]. >>> And also, if these Input/Output transport adapters and extensions for >>> MB/APP-M are in common to any device type, >>> cannot we move them down to carbon-device-mgt platform layer ? >>> >>> References : >>> [1] [Architecture] [IoT] Simplifying IoT device type plugin with a >>> descriptor >>> >>> Thanks, >>> Dilan. >>> >>> >>> *Dilan U. Ariyaratne* >>> Senior Software Engineer >>> WSO2 Inc. <http://wso2.com/> >>> Mobile: +94766405580 <%2B94766405580> >>> lean . enterprise . middleware >>> >>> >>> On Tue, Nov 22, 2016 at 11:58 AM, Ruwan Yatawara <ruw...@wso2.com> >>> wrote: >>> >>>> Hi All, >>>> >>>> In line with the changes that have been done to introduce the >>>> device-type descriptor in [1], I have gone ahead and done some refactoring >>>> to the existing plugins to standardise + make them self-contained. >>>> Following are a list of changes introduced. >>>> >>>>- Mobile Base plugin is no more : >>>> - Earlier windows/ios/android plugins relied on a shared >>>> component called the base plugin to include common functionality. >>>> This >>>> consisted of an OSGi component + a jaggery application. This made >>>> the 3 >>>> plugins interdependent. With the new structure, all 3 plugins would >>>> be >>>> self-contained and they would pack all the necessary backend >>>> functionality >>>> within themselves. The shared jaggery application has been broken >>>> down to 3 >>>> parts android-web-agent, windows-web-agent and ios-web-agent. Each >>>> application would get deployed and undeployed with the corresponding >>>> device >>>> type. >>>>- Each device type would be injecting the necessary UI units to a >>>>page/template unit contained inside of a the cdmf framework. >>>>- Each device type would be declaring its features/sensors and >>>>relevant tables as per [1] >>>>- Samples renamed to plugins and samples-deployer.xml renamed to >>>>plu
Re: [Architecture] Device Connectivity Graph for IoT Server & related concerns
Hi Ruwan, +1 for having audit tables for recording system activities. I also propose that we, take out the option for users to enable/disable data publishing from the agent side, and make it implicit. The agent by default makes a call to the server to send device information, instead of making the device make another call to DAS to provide location information, we can derive the location information at the server side from the device information payload and push it to DAS. +1 for above suggestion as well. IMO network calls between device & server should be minimal in-order to conserve power. Since we already have location data in device-info payload, there's no need for sending the same information to the DAS unless there is a need for real-time device tracking kinda scenario. Thanks, Harshan Liyanage EMM/IoT TG Mobile: *+94765672894* Email: hars...@wso2.com Blog : http://harshanliyanage.blogspot.com/ *WSO2, Inc. :** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Mon, May 8, 2017 at 12:35 PM, Ruwan Yatawara <ruw...@wso2.com> wrote: > Hi Everyone, > > I am working on $subject as part of the effort in trying to provide dig > down analytics for devices. > > The resulting graph would look something like the following (Please > disregard portion reading connected-unterminated), with the help of [1] and > will give an would indicate whether the device in question was able to get > back to IoT Server in a timely manner at the expected monitoring frequency > specified. > [image: Inline image 1] > *Source: [2]* > > Whilst attempting this I noticed that we do not have an auditing mechanism > in place to record important activities, as yet. If you take this flow, for > example, the monitoring frequency is something configurable and will change > from time to time. We need to know at which point the transition was made. > > Therefore I propose, that we come up with a central audit table to record > system activities, like updates to platform configurations, in a central > table. Each activity can have a logging code, and we can purge these > records from time to time, based on data growth. > > I also propose that we, take out the option for users to enable/disable > data publishing from the agent side, and make it implicit. The agent by > default makes a call to the server to send device information, instead of > making the device make another call to DAS to provide location information, > we can derive the location information at the server side from the device > information payload and push it to DAS. > > Please feel free to weigh in your thoughts. > > > [1] - https://bl.ocks.org/mbostock/4063318 > [2] - https://www.digi.com/resources/documentation/ > digidocs/90001150/reference/r_rep_device_connections.htm > > > Thanks and Regards, > > Ruwan Yatawara > > Associate Technical Lead, > WSO2 Inc. > > email : ruw...@wso2.com > mobile : +94 77 9110413 > blog : http://ruwansrants.blogspot.com/ > > https://500px.com/ruwan_ace > https://medium.com/@ruwanyatawara > > www: :http://wso2.com > > ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Methods to get currentSubject and currentStep in authentication script
+1 for 'context.currentKnownSubject' Thanks, Harshan Liyanage Mobile: *+94765672894* Email: hars...@wso2.com Blog: http://harshanliyanage.blogspot.com/ Medium: https://medium.com/@harshan.dll *WSO2, Inc.:** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Fri, Jul 20, 2018 at 12:22 PM Pulasthi Mahawithana wrote: > Hi Senthalan, > > Shall we have it something like, 'context.currentKnownSubject'? That would > imply that it returns the subject value from what it knows by the time of > execution. > > On Wed, Jul 18, 2018 at 4:30 PM Senthalan Kanagalingam > wrote: > >> Hi Harshan, >> >> No, when configuring the authentication steps, we can specify any step >> as subject-identifier-step. There will be no relationship >> between subject-identifier-step and last-authenticated-step. >> >> thanks >> >> On Mon, Jul 16, 2018 at 3:39 PM Harshan Liyanage >> wrote: >> >>> Hi Senthalan, >>> >>> According to your explanation, context.currentSubject will return >>> either *subject-identifier-step *or *last-authenticated-step. *Do we >>> have any direct relationship between the *subject-identifier-step *and >>> *last-authenticated-step? >>> *I'm just asking to clarify things. >>> >>> Regards, >>> >>> Harshan Liyanage >>> Mobile: *+94765672894* >>> Email: hars...@wso2.com >>> Blog: http://harshanliyanage.blogspot.com/ >>> Medium: https://medium.com/@harshan.dll >>> *WSO2, Inc.:** wso2.com <http://wso2.com/>* >>> lean.enterprise.middleware. >>> >>> >>> On Sat, Jul 14, 2018 at 2:38 PM Senthalan Kanagalingam < >>> sentha...@wso2.com> wrote: >>> >>>> Hi Harshan, >>>> >>>> On Fri, Jul 13, 2018 at 11:26 AM Harshan Liyanage >>>> wrote: >>>> >>>>> Hi Senthalan, >>>>> >>>>> What I understood by reading your description on the behavior of the >>>>> *context.currentSubject *method is that it always returns the subject >>>>> of the *last-completed-subject-identifier-step* rather than the >>>>> subject of the current subject identifier step. If my understanding is >>>>> correct, I suggest you change it to something more meaningful name such as >>>>> *context.lastSubject*. >>>>> >>>> >>>> No, if the* subject-identifier-step *is not completed, this method >>>> will return the subject of the lastly authenticated step. if >>>> *subject-identifier-step >>>> *is completed, it will return the subject of the >>>> *subject-identifier-step.* >>>> >>>> So, I think the name lastSubject can be meant as lastly authenticated >>>> steps' subject. >>>> >>>> thanks, >>>> >>>> >>>>> >>>>> I'm +1 with *context.currentStep.* >>>>> >>>>> Thanks, >>>>> >>>>> Harshan Liyanage >>>>> Mobile: *+94765672894* >>>>> Email: hars...@wso2.com >>>>> Blog: http://harshanliyanage.blogspot.com/ >>>>> Medium: https://medium.com/@harshan.dll >>>>> *WSO2, Inc.:** wso2.com <http://wso2.com/>* >>>>> lean.enterprise.middleware. >>>>> >>>>> >>>>> On Tue, Jul 10, 2018 at 9:39 PM Senthalan Kanagalingam < >>>>> sentha...@wso2.com> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I am working on to get the currently authenticated subject and >>>>>> currently executing step from the authentication script. Now, if the >>>>>> identity admin wants to get the authenticated subject, he/she has to know >>>>>> which step was set as the subject identifier step and have to call, >>>>>> "context.step[].subject". >>>>>> >>>>>> So, we have planned to implement a method as, >>>>>> >>>>>> *context.currentSubject * >>>>>> >>>>>> which will return the subject of the "subject identifier step", if >>>>>> that step is completed. Else return the subject of the last completed >>>>>> step. >>>>>> >>>>>> Another implementation is to have a method to get the currently >>>>>> exected method. Currently, identity
Re: [Architecture] Old access token cleanup when token generating, renewing and revoking.
Hi Nuwan, Thanks for your detailed clarifications. Both explanations are perfectly valid. Regards, Harshan Liyanage Mobile: *+94765672894* Email: hars...@wso2.com Blog: http://harshanliyanage.blogspot.com/ Medium: https://medium.com/@harshan.dll *WSO2, Inc.:** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Thu, Jul 19, 2018 at 8:35 PM Nuwan Dias wrote: > > > On Wed, Jul 18, 2018 at 9:09 PM Harshan Liyanage wrote: > >> Hi all, >> >> @Nuwan: That's why I suggested having a configurable cron expression so >> that users can configure the task to run on an optimal schedule instead of >> blocking vital functionalities. Also in that way, we could do a batch >> deletes and insertions instead of single rows. For example, they could let >> it run every mid-night so then the access token tables won't grow for >> millions of records and not affecting the user traffic. >> > > Our objective is to run this clean up process without anyone having to > configure anything. Even today we have instructions and scripts to clean up > the tables. But in practice, nobody even notices this and never does these > kind of stuff until they hit an issue and consult us through support. So if > we design this in such a way that someone has to turn on something or > configure something, I am positive our efforts will be in waste. > >> >> @Nalaka: You could let the task to run only on a manager node in a >> distributed setup using a configuration. >> > > There is no such thing as a "manager" node in our architecture. All nodes > are equal. Plus, we cannot introduce such changes as well due to numerous > complexities such as how to HA the manager node, too many varying > configurations, etc. > >> >> BTW that was just a suggestion. It doesn't mean I'm -1 on this proposed >> design. :) >> >> Thanks, >> >> Harshan Liyanage >> Mobile: *+94765672894* >> Email: hars...@wso2.com >> Blog: http://harshanliyanage.blogspot.com/ >> Medium: https://medium.com/@harshan.dll >> *WSO2, Inc.:** wso2.com <http://wso2.com/>* >> lean.enterprise.middleware. >> >> >> On Wed, Jul 18, 2018 at 8:51 PM Nalaka Senarathna >> wrote: >> >>> hi harshan, >>> >>> Also if there are multiple nodes then those nodes also may attempt to >>> clean up the same access token at the same time. >>> >>> related mail thread:[1] >>> [1]Access token Table cleaning and keeping the access token data for >>> future purposes >>> >>> regards. >>> >>> >>> On Wed, Jul 18, 2018 at 8:28 PM, Nuwan Dias wrote: >>> >>>> A periodic task won't work for this because when the system runs on >>>> tables with millions of records, the task will lock the table for the clean >>>> up process. This will impact other queries being executed on the table and >>>> hence block user flows. >>>> >>>> On Wed, Jul 18, 2018 at 6:17 AM Harshan Liyanage >>>> wrote: >>>> >>>>> Hi Nalaka, >>>>> >>>>> You could do the same with a configurable periodic task instead of >>>>> modifying existing token request flows. What you have to do is to >>>>> implement >>>>> the token cleanup feature as a periodic task which scans the token related >>>>> tables and move EXPIRED, INACTIVE and REVOKED tokens to the audit >>>>> table. There will be a configuration to configure the CRON expression for >>>>> that task. >>>>> >>>>> WDYT? >>>>> >>>>> Thanks, >>>>> >>>>> Harshan Liyanage >>>>> Mobile: *+94765672894* >>>>> Email: hars...@wso2.com >>>>> Blog: http://harshanliyanage.blogspot.com/ >>>>> Medium: https://medium.com/@harshan.dll >>>>> *WSO2, Inc.:** wso2.com <http://wso2.com/>* >>>>> lean.enterprise.middleware. >>>>> >>>>> >>>>> On Wed, Jul 18, 2018 at 2:45 PM Nalaka Senarathna >>>>> wrote: >>>>> >>>>>> >>>>>> A solution for the access token table filled up with EXPIRED, >>>>>> INACTIVE and REVOKED tokens in the Access token table, old access token >>>>>> can >>>>>> move to the Audit table when the new token is generating, renewing or >>>>>> token >>>>>> revoking. >>>>>> >>>>>> >>&g
Re: [Architecture] Methods to get currentSubject and currentStep in authentication script
Hi Senthalan, According to your explanation, context.currentSubject will return either *subject-identifier-step *or *last-authenticated-step. *Do we have any direct relationship between the *subject-identifier-step *and *last-authenticated-step? *I'm just asking to clarify things. Regards, Harshan Liyanage Mobile: *+94765672894* Email: hars...@wso2.com Blog: http://harshanliyanage.blogspot.com/ Medium: https://medium.com/@harshan.dll *WSO2, Inc.:** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Sat, Jul 14, 2018 at 2:38 PM Senthalan Kanagalingam wrote: > Hi Harshan, > > On Fri, Jul 13, 2018 at 11:26 AM Harshan Liyanage > wrote: > >> Hi Senthalan, >> >> What I understood by reading your description on the behavior of the >> *context.currentSubject *method is that it always returns the subject of >> the *last-completed-subject-identifier-step* rather than the subject of >> the current subject identifier step. If my understanding is correct, I >> suggest you change it to something more meaningful name such as >> *context.lastSubject*. >> > > No, if the* subject-identifier-step *is not completed, this method will > return the subject of the lastly authenticated step. if > *subject-identifier-step > *is completed, it will return the subject of the > *subject-identifier-step.* > > So, I think the name lastSubject can be meant as lastly authenticated > steps' subject. > > thanks, > > >> >> I'm +1 with *context.currentStep.* >> >> Thanks, >> >> Harshan Liyanage >> Mobile: *+94765672894* >> Email: hars...@wso2.com >> Blog: http://harshanliyanage.blogspot.com/ >> Medium: https://medium.com/@harshan.dll >> *WSO2, Inc.:** wso2.com <http://wso2.com/>* >> lean.enterprise.middleware. >> >> >> On Tue, Jul 10, 2018 at 9:39 PM Senthalan Kanagalingam < >> sentha...@wso2.com> wrote: >> >>> Hi all, >>> >>> I am working on to get the currently authenticated subject and currently >>> executing step from the authentication script. Now, if the identity admin >>> wants to get the authenticated subject, he/she has to know which step was >>> set as the subject identifier step and have to call, >>> "context.step[].subject". >>> >>> So, we have planned to implement a method as, >>> >>> *context.currentSubject * >>> >>> which will return the subject of the "subject identifier step", if that >>> step is completed. Else return the subject of the last completed step. >>> >>> Another implementation is to have a method to get the currently exected >>> method. Currently, identity admin has to specify the step number in order >>> to get the details. "context.step[]". This will affect the >>> reusability of the code. >>> >>> With this new implementation, the identity admin can use, >>> >>> *context.currentStep* >>> >>> which will return the executing step. >>> >>> Please share your comments on the naming of the methods. >>> >>> thanks, >>> Senthalan. >>> -- >>> >>> *Senthalan Kanagalingam* >>> *Software Engineer - WSO2 Inc.* >>> *Mobile : +94 (0) 77 18 77 466* >>> <http://wso2.com/signature> >>> ___ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >> ___ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> > > > -- > > *Senthalan Kanagalingam* > *Software Engineer - WSO2 Inc.* > *Mobile : +94 (0) 77 18 77 466* > <http://wso2.com/signature> > ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Stream Processor HA improved archectecture
Hi Damith, According to this architecture, the passive node will only fetch event states and filter events only once it is active. This model might exhaust the queue 2 very soon depending on the rate of events. What if we could add another async task to check the state and filter events even if the node is passive? Thanks, Harshan Liyanage Mobile: *+94765672894* Email: hars...@wso2.com Blog: http://harshanliyanage.blogspot.com/ Medium: https://medium.com/@harshan.dll *WSO2, Inc.:** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Fri, Jul 13, 2018 at 9:01 AM Tishan Dahanayakage wrote: > @Anoukh, > We can configure a LB in front to handle failover. Actually that is the > correct solution rather than publishing to both. But I think we need to > figure out how to handle incoming databridge traffic as it cannot be load > balanced. Yes we can utilize client side load balancing and directly > publish to SP nodes. But then we will need to shutdown thrift/binary > receiving servers of passive node for the failover to happen. > > @Damith, > I have below questions/suggestions. > 1. What is the expected behavior when queue is full in active node? > First of all I suppose this is a in-memory queue. This is something we > should think clearly and implement as even now we are facing quite a number > of issues where server is stalled because queues are getting filled. When > queue in active node is full > - We can't let receiving threads to get block on the queue as it will > cause server to freeze. > - At the same time We can't also drop the event if exactly once processing > or zero event drop is enforced. > - Also we can't keep adding stuff to queue as it will go OOM. > So IMO we need a way to persist the queue. In other words we need a proper > broker queue to solve this issue. If user is fine with event loss then We > can live with in-memory queue. > > 2. We need a automated purging task to clean queue2 to after every state > persistance done by active node. > > /Tishan > > On Thu, Jul 12, 2018 at 10:41 PM, Anoukh Jayawardena > wrote: > >> Hi Damith, >> >> Just a few clarifications on the new architecture. >> I'm assuming that "queue1" and "queue2" are actually per Source? Which >> means if we have 3 sources, we'll have 3 queues in active node and 3 queues >> in passive node? Does that mean there will be 3 threads working in async? >> Also, the main difference with the current HA architecture is that both >> nodes will not receive all the events. So at the moment the active node >> goes down, how does the passive node start receiving events? Will we be >> handling this or should the cluster be fronted by a separate load balancer? >> >> Thanks, >> Anoukh >> >> On Thu, Jul 12, 2018 at 8:34 PM Damith Wickramasinghe >> wrote: >> >>> Hi all, >>> >>> We are in the process of refactor/improve the existing HA architecture >>> due to various concerns found. >>> >>> Below is the high level design came up with. We will provide more >>> in-depth details as the implementation carries on. >>> >>> >>> >>> >>> As per above at a given point of time there will only be a one active >>> node. Passive node will not consume any events. Electing active node will >>> work as per current architecture via cluster coordination. >>> A thread will work in the active node to put the data into a >>> queue(queue1) and same thread will then publish the events to >>> outside.Reason for having a queue here because to send events >>> asynchronously to passive node. Here we are going to send events to passive >>> node via TCP. (This we need to decide) Active node will persist the state >>> periodically to the database. >>> >>> When active node goes down via cluster coordination passive node will >>> become active. When active it will get the state from database(to sync the >>> state) with latest event timestamp , and filter events from queue 2 (in >>> order to stop processing already processed events) and send them out. Then >>> open the ports in the newly active node and start receiving events from >>> sources. In this architecture also at least one processing will be done. >>> >>> Nisala please add anything I missed. >>> >>> Regards, >>> Damith >>> >>> >>> >>> >>> >>> >>> -- >>> Senior Software Engineer >>> WSO2 Inc.; http://wso2.com >>> <http://www.google.com/url?q=http%3A%2F%2Fwso2.com=D=1=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg> &g
Re: [Architecture] Methods to get currentSubject and currentStep in authentication script
Hi Senthalan, What I understood by reading your description on the behavior of the *context.currentSubject *method is that it always returns the subject of the *last-completed-subject-identifier-step* rather than the subject of the current subject identifier step. If my understanding is correct, I suggest you change it to something more meaningful name such as *context.lastSubject*. I'm +1 with *context.currentStep.* Thanks, Harshan Liyanage Mobile: *+94765672894* Email: hars...@wso2.com Blog: http://harshanliyanage.blogspot.com/ Medium: https://medium.com/@harshan.dll *WSO2, Inc.:** wso2.com <http://wso2.com/>* lean.enterprise.middleware. On Tue, Jul 10, 2018 at 9:39 PM Senthalan Kanagalingam wrote: > Hi all, > > I am working on to get the currently authenticated subject and currently > executing step from the authentication script. Now, if the identity admin > wants to get the authenticated subject, he/she has to know which step was > set as the subject identifier step and have to call, > "context.step[].subject". > > So, we have planned to implement a method as, > > *context.currentSubject * > > which will return the subject of the "subject identifier step", if that > step is completed. Else return the subject of the last completed step. > > Another implementation is to have a method to get the currently exected > method. Currently, identity admin has to specify the step number in order > to get the details. "context.step[]". This will affect the > reusability of the code. > > With this new implementation, the identity admin can use, > > *context.currentStep* > > which will return the executing step. > > Please share your comments on the naming of the methods. > > thanks, > Senthalan. > -- > > *Senthalan Kanagalingam* > *Software Engineer - WSO2 Inc.* > *Mobile : +94 (0) 77 18 77 466* > <http://wso2.com/signature> > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
