Re: [Architecture] Deployable Artifact Model

[+ arch@] Hi, IMO, we have to go with separate artifacts, for the easiness of maintainability. for eg: if we have separate artifacts (say for projects and analyzes), - One can easily add and remove analyzes at any time from a project, just by adding/deleting the file corresponds to that

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

On Fri, Apr 1, 2016 at 10:18 AM, Isuru Haththotuwa wrote: > > > On Fri, Apr 1, 2016 at 12:45 AM, Imesh Gunaratne wrote: > >> >> >> On Thu, Mar 31, 2016 at 7:56 PM, Isuru Haththotuwa >> wrote: >> >>> >>> On Thu, Mar 31, 2016 at 4:47 PM, Imesh

Re: [Architecture] Application Server 6.0.0 - Architecture

All the valves and extensions are part of tomcat. We can show the runtime engine separately and we do not need to show the request flow on the same diagram but focus only on the core components and where it fits in the architecture. The valve names should be correctly named. What we have is SAML

Re: [Architecture] Application Server 6.0.0 - Architecture

Hi Manuri, On Fri, Apr 1, 2016 at 9:23 AM, Manuri Amaya Perera wrote: > Hi Kalpa, > > In the request flow diagram, why are tomcat valves orthogonal to the other > valves? > > "Tomcat Valves" represents that the above are Tomcat valves, the boarder of "Tomcat valves" makes it

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

On Fri, Apr 1, 2016 at 12:45 AM, Imesh Gunaratne wrote: > > > On Thu, Mar 31, 2016 at 7:56 PM, Isuru Haththotuwa > wrote: > >> >> On Thu, Mar 31, 2016 at 4:47 PM, Imesh Gunaratne wrote: >> >>> >>> On Thu, Mar 31, 2016 at 2:05 PM, Isuru

Re: [Architecture] Adding RNN to WSO2 Machine Learner

Hi, I have organized a review on Monday (4th of April). Thanks On Thu, Mar 31, 2016 at 3:21 PM, Srinath Perera wrote: > Please setup a review. Shall we do it monday? > > On Thu, Mar 31, 2016 at 2:15 PM, Thamali Wijewardhana > wrote: > >> Hi, >> >> we have

Re: [Architecture] ESB Debugger - Representing wire message debugging at the visual editor

Hi, I was able to simply show the wire log (what is just written to the wire) as a property when a debug point get hits, But in that case there are some drawbacks, such as we can't see back end call wire logs because we can't set debug point just after back end call happens, (we can see back end

Re: [Architecture] Application Server 6.0.0 - Architecture

Hi Kalpa, In the request flow diagram, why are tomcat valves orthogonal to the other valves? Thanks. Manuri On Thu, Mar 31, 2016 at 3:53 PM, Kalpa Welivitigoda wrote: > Hi all, > > WSO2 Application Server 6.0.0 is based on Apache Tomcat 8.0. To > add/enhance functionality, we

Re: [Architecture] Support Efficient Cross Tenant Analytics in DAS

Use case come from IoT and APIM, and may be others. And it will be a common use case due to our cloud. On Thu, Mar 31, 2016 at 3:55 PM, Anjana Fernando wrote: > Hi Srinath, > > I'm not sure if this is something we would have to "fix". It was a clear > design decision we took in

Re: [Architecture] Data Isolation level for Data from APIM and IoT? Tenant vs. User

Hi Frank, I agree on your concerns. However, reason we are though of doing this is relationship between data item and a user is not 1-1, but rather complicated. e.g. Problem: Bob writes a API and publish it API manager. Then Alice subscribes to the API and write an mobile APP. Charlie uses the

Re: [Architecture] Support Efficient Cross Tenant Analytics in DAS

Hi, Having to run spark queries for each tenant can be very expensive with large number of tenants, but in terms of data isolation current design is better I believe. If we can come up with a good to design for supporting tenant level data isolation, this is something we can do indeed. However,

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

On Thu, Mar 31, 2016 at 7:56 PM, Isuru Haththotuwa wrote: > > On Thu, Mar 31, 2016 at 4:47 PM, Imesh Gunaratne wrote: > >> >> On Thu, Mar 31, 2016 at 2:05 PM, Isuru Haththotuwa >> wrote: >> >>> >>> [2]. >>> FROM wso2am:1.10.0 >>> MAINTAINER

Re: [Architecture] Circuit Breaker Pattern for MSF4J

The blog post has been removed. Sorry for all the confusion. This was only done as part of the agreement we had during last week's meeting to demonstrate certain features such as Spring support, JPA support, support for patterns etc. in order to help developers understand how to implement certain

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

On Thu, Mar 31, 2016 at 4:47 PM, Imesh Gunaratne wrote: > > > On Thu, Mar 31, 2016 at 2:05 PM, Isuru Haththotuwa > wrote: > >> >> [2]. >> FROM wso2am:1.10.0 >> MAINTAINER isu...@wso2.com >> >> COPY artifacts/ /mnt/wso2-artifacts/carbon-home >> > We are not using

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

On Thu, Mar 31, 2016 at 5:23 PM, Gayan Gunarathne wrote: > > > On Thu, Mar 31, 2016 at 2:05 PM, Isuru Haththotuwa > wrote: > >> Earlier the PaaS team had a discussion in architecture list thread [1] to >> create a base Docker image and a profile Docker image

Re: [Architecture] Circuit Breaker Pattern for MSF4J

On Thu, Mar 31, 2016 at 5:03 PM, Frank Leymann wrote: > Yes, all the stability patterns (that Nygard describes, the circuit > breaker being just one of them) > is not associated with microservices, but applies to all distributed > applications. In fact, Nygard's > book has been

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

On Thu, Mar 31, 2016 at 4:57 PM, Chamila De Alwis wrote: > @Sajith, > > Publishing WSO2 products on the Docker official repository will be a > process which involves pushing our Dockerfiles to > docker-library/official-images [1], and it will involve PRs for them to be >

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

On Thu, Mar 31, 2016 at 2:05 PM, Isuru Haththotuwa wrote: > Earlier the PaaS team had a discussion in architecture list thread [1] to > create a base Docker image and a profile Docker image extending the base > Docker image per WSO2 product. This was done for the ease of the

Re: [Architecture] Circuit Breaker Pattern for MSF4J

Yes, all the stability patterns (that Nygard describes, the circuit breaker being just one of them) is not associated with microservices, but applies to all distributed applications. In fact, Nygard's book has been published in 2007, lng before the microservice discussion came up ;-) Applying

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

@Sajith, Publishing WSO2 products on the Docker official repository will be a process which involves pushing our Dockerfiles to docker-library/official-images [1], and it will involve PRs for them to be updated. IMO this would interfere with our ability to have control over the Dockerfile

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

On Thu, Mar 31, 2016 at 2:05 PM, Isuru Haththotuwa wrote: > > [2]. > FROM wso2am:1.10.0 > MAINTAINER isu...@wso2.com > > COPY artifacts/ /mnt/wso2-artifacts/carbon-home > Shouldn't it better to use a simple folder structure like "/usr/local/wso2/wso2/" instead of above? Apache

Re: [Architecture] Data Isolation level for Data from APIM and IoT? Tenant vs. User

Sorry for jumping into the discussion late (or even too late). I try to understand the discussion by drawing analogies to DBMS - maybe that's wrong and I miss the point... If I am right, what you decided in the meeting is fully in line with what DBMS are doing :-) In Srinath's summary, list item

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

It seems that is the recommended notation, not only for some products, but for all the 'official' repositories. If we want to make our images as official images, according to [1] and [2] we should not provide any organization name. For esb it wil be like https://hub.docker.com/_/wso2esb For

Re: [Architecture] Support Efficient Cross Tenant Analytics in DAS

Hi Srinath, I'm not sure if this is something we would have to "fix". It was a clear design decision we took in order to isolate the tenant data, in order for others not to access other tenant's data. So also in Spark virtual tables, it will directly map to their own analytics tables. If we

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

Hi Sajith, There are repositories on Dockerhub without organizations, such as mysql [1] and tomcat [2] which are official repositories. When the images are pushed to DockerHub, the organization name can be used as "wso2". Furthermore, the products are WSO2 prefixed (WSO2MB vs MB) so the name

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

Hi Sajith, By default, it creates image as wso2as:5.3.0. And we have options to add the organization name, where by executing the script as "*./build.sh -v 5.3.0 -o *wso2", it will create the image name as wso2/wso2as:5.3.0. Regards, Vishanth On Thu, Mar 31, 2016 at 3:25 PM, Sajith Kariyawasam

Re: [Architecture] [Docker] Creating a Single Docker Image for a Product

It seems the image name format we have now is wso2as:5.3.0. Shouldn't it be like wso2/as:5.3.0 ? Otherwise, once we push to dockerhub in future, we need to have separate user accounts for each product which is not feasible. On Thu, Mar 31, 2016 at 2:05 PM, Isuru Haththotuwa

[Architecture] [Docker] Extensible approach to building Docker images

Hi, As described in the Isuru's email [1], wso2/dockerfiles[2] structure was simplified to enable a single Docker image per WSO2 product. At the same time, the approach that was used to build the Docker images was changed from Puppet to an extensible approach. Earlier only a "puppet apply"

Re: [Architecture] [C5] Artifact deployment coordination support for C5

Hi All, Do we really want artifact deployment coordination in C5? What is preventing us to build the new image with the new version of artifacts and let the k8s take care of deployment? Cheers, Ruwan On Wed, Mar 30, 2016 at 2:54 PM, Isuru Haththotuwa wrote: > Hi Kasun, > > On

Re: [Architecture] Adding RNN to WSO2 Machine Learner

Hi, we have created a spark program to prove the feasibility of adding the RNN algorithm to machine learner. This program demonstrates all the steps in machine learner: Uploading a dataset Selecting the hyper parameters for the model Creating a RNN model using data and training the model

[Architecture] [Docker] Creating a Single Docker Image for a Product

Earlier the PaaS team had a discussion in architecture list thread [1] to create a base Docker image and a profile Docker image extending the base Docker image per WSO2 product. This was done for the ease of the users of wso2 Dockerfiles. More details can be found in the mentioned thread. However,

Re: [Architecture] RFC:Security Challenges in Analytics Story

Please see "Data Isolation level for Data from APIM and IoT? Tenant vs. User" for decisions --Srinath On Fri, Mar 25, 2016 at 10:06 AM, Srinath Perera wrote: > As per meeting ( Sanjiva, Shankar, Sumedha, Anjana, Miyuru, Seshika, Suho, > Nirmal, Nuwan) > > We need APIM and IOT

Re: [Architecture] Data Isolation level for Data from APIM and IoT? Tenant vs. User

We had a meeting. Participants: Sanjiva, Sumedha, NuwanD, Prabath, Srinath (Prabath please list others joined from Trace) Problem: Bob writes a API and publish it API manager. Then Alice subscribes to the API and write an mobile APP. Charlie uses the mobile App, which results in an API call. We

Re: [Architecture] Circuit Breaker Pattern for MSF4J

Agreed. However I had understood that the circuit breaker pattern was advocated primarily for service clients in MSA (and of course it has nothing do with being micro). The general story of better failure handling applies to all code and is of course not MSA specific. Anyway .. Sample is fine.

Re: [Architecture] Supporting Carbon Metrics on Carbon 5 based products

Hi Azeez, I meant the same thing as you. I should have explained it more clearly. :) So, I'll break metrics into 2 repositories. One repository will be having the core Metrics features, which *will be used* by MSF4J. Other metrics related repository will have the REST API, written with MSF4J.

[Architecture] Support Efficient Cross Tenant Analytics in DAS

Hi Anjana, Currently we keep different Hbase/ RDBMS table per tenant. In multi-tenant, environment, this is very expensive as we will have to run a query per tenant. How can we fix this? e.g. if we keep tenant as field in the table, that let us do a "group by". --Srinath --