Re: [Architecture] [AF] Adding ESB Applications to App Factory

[Gayan Dhanushka]

Hi all,

We have completed the initial dev work related to ESB app type in App
Factory. We have completed the following during the last couple of weeks.


   - The issue that we observed as described at the end of the previous
   update (looking for a JKS in a path called /root/.keystore ) was due to the
   https configuration in the catalina-server.xml not having the keystore
   path. Fixed that issue by adding the keystore path.
   - Cartridge agent was failing to clone the git repositories due to the
   certificate validation failure. Fixed that with adding GIT_SSL_NO_VERIFY=1
   to the init script of the docker image.
   - Got the docker images to spawn new pods properly by solving the above
   issues.
   - Changed the maven multi-module archetype to correct the deployment of
   the ESB synapse configuration in the archetype.
   - Tested the application creation up until deployment of the car files.
   Verified that this flaw was working fine for the trunk version of the ESB
   app type. But found issues when tried to create a version where the
   artifacts.xml in the car file did not get updated with the new version of
   the proxy service artifact. Fixed this issue in the
   AppVersionStrategyExecuter where the artifact.xml was not updated properly
   through the code.
   - Tested the application promotion and demotion. This was working fine
   with the WSDL files and the endpoints were also getting copied across the
   registries in different environments.
   - Provided a mechanism to add new endpoints and edit existing endpoints.
   This was done using the properties section under the Runtime Configs
   section. Endpoints can be newly added as properties and the existing
   properties can be viewed and edited.
   - A new page is created to view the artifacts inside the car file once
   the launch button is clicked.

What is left to make the feature comprehensive is to provide a dev studio
integration for the ESB app type and write a hook to validate whether the
synapse artifacts are in proper format. We will continue this work in the
upcoming weeks.
Regards


On Fri, Oct 16, 2015 at 7:54 PM, Gayan Dhanushka  wrote:

> Hi all,
>
> I am continuing the work from the point Danushka has left us with. This
> week I have been doing the following.
>
>
>- Setting us a new environment to test the ESB runtime.
>- Writing json files to deploy the ESB cartriges and the ESB
>applications for the three environments. Deploying the json files to the
>private paas and creating the cartridges and applications in it.
>- Testing the ESB docker image on a Kubernetes cluster. *-* Found some
>issues while testing. Initially a pod was spawned but the ESB pack was not
>getting unzipped and fixed it in the init.sh. After fixing the above
>observed that the ESB instance was not getting started up automatically.
>Found out that the JAVA environment variables are not getting set and fixed
>it. Then found out that the ESB startup python scripts were missing in the
>docker image and added them to the plugins folder of the ESB cartridge.
>Finally got the image to spin up the ESB instance at the pod spawning time.
>
> Need to fix a small issue observed while the ESB server was getting
> started. The issue was related to tomcat getting started. It was looking
> for a JKS in a path called /root/.keystore .
>
> I will proceed with the rest of the work next week.
>
> Regards
>
> Gayan
>
>
>
>
> On Sun, Oct 4, 2015 at 9:26 PM, Danushka Fernando 
> wrote:
>
>> Hi Guys
>> First of all sorry for the late update. We compared the dbscripts of ESB
>> 4.8.1 and ESB 4.9.0 and realized they are similar. So I believe we can
>> share the same databases across 4.2.x Products which we have in cloud and
>> ESB 4.9.0. But still if we need to use the components that we developed for
>> AS we will need to maintain a separate version which is based on 4.4.x
>> kernel. I guess that's simple development work.
>>
>> So here is the update about what has been done up to now and what has to
>> be done further.
>>
>>- Developing the apptype for car to create a sample app with
>>functionalities build, deploy and promote (Copy to artifact repo). --
>>*Done.*
>>- Developing the docker image for ESB 4.9.0 -- *Done, Need to be
>>tested.*
>>- Develop json files and scripts to create and deploy the cartridges
>>and  execute them at the setup time. (We can just copy the files related 
>> to
>>Appserver and we can change the parameters) -- *Not Done.*
>>- Need to test launching and we need to decide what to show when we
>>launch a ESB application and we need to generate the launch url based on
>>that. For this we will need to develop some components which will pass the
>>deployment status from ESB to AF similar to what we developed for web
>>applications and services deployed in app server. -- *Not Done.*
>>- Dev studio 

Re: [Architecture] [AF] Adding ESB Applications to App Factory

[Danushka Fernando]

Hi Guys
First of all sorry for the late update. We compared the dbscripts of ESB
4.8.1 and ESB 4.9.0 and realized they are similar. So I believe we can
share the same databases across 4.2.x Products which we have in cloud and
ESB 4.9.0. But still if we need to use the components that we developed for
AS we will need to maintain a separate version which is based on 4.4.x
kernel. I guess that's simple development work.

So here is the update about what has been done up to now and what has to be
done further.

   - Developing the apptype for car to create a sample app with
   functionalities build, deploy and promote (Copy to artifact repo). --
   *Done.*
   - Developing the docker image for ESB 4.9.0 -- *Done, Need to be tested.*
   - Develop json files and scripts to create and deploy the cartridges and
execute them at the setup time. (We can just copy the files related to
   Appserver and we can change the parameters) -- *Not Done.*
   - Need to test launching and we need to decide what to show when we
   launch a ESB application and we need to generate the launch url based on
   that. For this we will need to develop some components which will pass the
   deployment status from ESB to AF similar to what we developed for web
   applications and services deployed in app server. -- *Not Done.*
   - Dev studio integration for App Factory ESB application with applying
   rules previously discussed. -- *Not Done.*
   - Git hook to check whether committed files are according to the rules..
   -- *Not Done.*




Thanks & Regards
Danushka Fernando
Senior Software Engineer
WSO2 inc. http://wso2.com/
Mobile : +94716332729

On Sun, Oct 4, 2015 at 7:43 AM, Kasun De Silva  wrote:

> Hi Manjula/Danushka and All,
>
> When I was reading through the options listed by Danushka, I got the
> preference to #1 as well,
>
> Go with ESB 4.8.1 for now. (Later we need to upgrade all the products to
>> 4.4.x kernel) Backport the fix for logging to 4.8.1 if possible. (@ESB team
>> is this possible?)
>>
>
> Because #2 and #3 would add much complexity to the cloud deployment. I
> think we are at a state that we should consider the stability of the cloud
> deployment because we are planing to go production late this year. So for
> the sake of simplicity and the stability of the we should not introduce
> anything that is adding any complexity cloud deployment, So my personal
> preference would be go with #1.
>
> And I believe we can do an upgrade later, and first thing should be we
> should get ESB apptype running to cloud deployment. So I think going with
> ESB 4.8.1 would be the ideal case.
>
> But in the previous mail Kasun mentioned that the iPass won't work with
> 4.8.1 and other drawbacks with the option #1. So I think we better have a
> session to analyse the options we have  with one or two ESB team members
> and select the best option for us.
>
> WDYT?
>
>
> Thanks,
> Kasun
>
> *Kasun de Silva*
> Software Engineer | *WSO2 Inc.*; http://wso2.com
> lean.enterprise.middleware
>
> email   : kas...@wso2.com
> mobile : +94 77 794 4260
>
>
> On Thu, Oct 1, 2015 at 10:38 AM, Kasun Indrasiri  wrote:
>
>> Hi,
>>
>> Not having 4.9 in the ESB as a service will be a huge drawback.. IMO.
>> There are quite a lot of changes that we specifically did for 4.9 to fix
>> the behavior in MT mode etc. (inbound EP, tasks coordination etc.). So, I
>> don't think going back to 4.8.1 is the best option we have. Anyway, iPaaS
>> work also needs ESB 4.9, so going back to 4.8.1 won't work for iPaaS.
>>
>> thanks,
>> Kasun
>>
>> On Thu, Oct 1, 2015 at 10:04 AM, Manjula Rathnayake 
>> wrote:
>>
>>> Hi all,
>>>
>>> I am +1 for #1 due to the simplicity we get in cloud deployment.
>>>
>>> @Nadeeshaan, can we get the car based logging improvement to ESB 4.8.1?
>>>
>>> thank you.
>>>
>>> On Wed, Sep 30, 2015 at 5:32 PM, Danushka Fernando 
>>> wrote:
>>>
 Hi All,

 I am working on adding ESB Application to App Factory. Currently all
 application type related code is developed and sample code also done. While
 looking into adding ESB container faced an issue.

 Currently AF dev setups and App Cloud deployed all the products which
 are based on Carbon 4.2.0. But ESB 4.9.0 contains a feature to add log
 filtering based on Capp names. But AFAIK ESB 4.9.0 is based on carbon
 4.4.x. So we cannot mount existing databases to ESB 4.9.0.


 So the possible solutions are as below.



1.

Go with ESB 4.8.1 for now. (Later we need to upgrade all the
products to 4.4.x kernel) Backport the fix for logging to 4.8.1 if
possible. (@ESB team is this possible?)
2.

Deploy ESB 4.9.0 with new database schemas and when we create
tenants we will need to create tenants in ESB as well and also when we
create resources we need to replicate them in ESB as well.
3.

   

Re: [Architecture] [AF] Adding ESB Applications to App Factory

[Kasun Indrasiri]

Hi,

Not having 4.9 in the ESB as a service will be a huge drawback.. IMO. There
are quite a lot of changes that we specifically did for 4.9 to fix the
behavior in MT mode etc. (inbound EP, tasks coordination etc.). So, I don't
think going back to 4.8.1 is the best option we have. Anyway, iPaaS work
also needs ESB 4.9, so going back to 4.8.1 won't work for iPaaS.

thanks,
Kasun

On Thu, Oct 1, 2015 at 10:04 AM, Manjula Rathnayake 
wrote:

> Hi all,
>
> I am +1 for #1 due to the simplicity we get in cloud deployment.
>
> @Nadeeshaan, can we get the car based logging improvement to ESB 4.8.1?
>
> thank you.
>
> On Wed, Sep 30, 2015 at 5:32 PM, Danushka Fernando 
> wrote:
>
>> Hi All,
>>
>> I am working on adding ESB Application to App Factory. Currently all
>> application type related code is developed and sample code also done. While
>> looking into adding ESB container faced an issue.
>>
>> Currently AF dev setups and App Cloud deployed all the products which are
>> based on Carbon 4.2.0. But ESB 4.9.0 contains a feature to add log
>> filtering based on Capp names. But AFAIK ESB 4.9.0 is based on carbon
>> 4.4.x. So we cannot mount existing databases to ESB 4.9.0.
>>
>>
>> So the possible solutions are as below.
>>
>>
>>
>>1.
>>
>>Go with ESB 4.8.1 for now. (Later we need to upgrade all the products
>>to 4.4.x kernel) Backport the fix for logging to 4.8.1 if possible. (@ESB
>>team is this possible?)
>>2.
>>
>>Deploy ESB 4.9.0 with new database schemas and when we create tenants
>>we will need to create tenants in ESB as well and also when we create
>>resources we need to replicate them in ESB as well.
>>3.
>>
>>Add ESB similar to a single tenant cartridge (ESB per version) and
>>point it to databases created from 4.4.x database schemas. These databases
>>will be per container. Issues with this are we need to replicate the
>>resources we create in ESB containers and if container get killed we will
>>lose the data. And implementation will be much more complex.
>>
>>
>> I am +1 for the Solution #1 since later in the day we need to upgrade all
>> products and it’s less complex in deployment. And also in the worst case if
>> we can’t patch with the logging feature then log download function won’t be
>> available immediately to ESB application types.
>>
>> Thanks & Regards
>> Danushka Fernando
>> Senior Software Engineer
>> WSO2 inc. http://wso2.com/
>> Mobile : +94716332729
>>
>
>
>
> --
> Manjula Rathnayaka
> Associate Technical Lead
> WSO2, Inc.
> Mobile:+94 77 743 1987
>



-- 
Kasun Indrasiri
Software Architect
WSO2, Inc.; http://wso2.com
lean.enterprise.middleware

cell: +94 77 556 5206
Blog : http://kasunpanorama.blogspot.com/
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture