Re: [Dev] [AppCloud] Detecting idle containers in appcloud
Hi All, After installing Heapster with the custom docker image [1], I was able to get historical data of 1 hour. But these data have to be examined to understand which data can be used to detect the idle containers in Kubernetes cluster. So one approach would be to store these data periodically in an external database, examine the data and get which data would give the needed information to detect the idle containers. [1] https://hub.docker.com/r/malmee/malmee2heapster/ Thank you. On Tue, Apr 19, 2016 at 12:09 PM, Imesh Gunaratne wrote: > > > On Tue, Apr 19, 2016 at 11:58 AM, Malmee Weerasinghe > wrote: > >> Hi All, >> >> As there is a default database 'InfluxDB' that comes with 'Heapster' >> using the same database would be a better approach than an external >> database. >> > > Yes IMO it is better to use InfluxDB for this. > > Thanks > >> >> Default Heapster gives only up to 15 minutes of historical data for any >> Container. Would using a custom Heapster docker image be a better approach >> on this? >> >> In Heapster Repository, Heapster.go file contains these configurations. >> >>- stats_resolution (Deafult 1*time.Second) The resolution at which >>heapster will retain stats >>- sink_frequency (Default 10*time.Second) Frequency at which data >>will be pushed to sinks >>- cache_duration (Default 4*time.Minute) The total duration of the >>historical data that will be cached by heapster >>- use_model (Default true) When true, the internal model >>representation will be used >>- model_resolution (Default 1*time.Minute) The resolution of the >>time series stored in the model. >>- model_frequency (Default 145*time.Second) Frequency at which model >>will be updated. >> >> >> >> * 1 second < stats_resolution < model_resolution < model_frequency >> sink_frequency < cache_duration* >> >> If these configuration are changed as below, build a custom docker image >> and use that docker image instead of default Heapster image >> Heapster:canary, would that be a better approach on this? Or is there any >> other approaches for this? >> >>- stats_resolution1*time.Hour >>- sink_frequency 2*time.Hour >>- cache_duration 3*time.Hour >>- use_modeltrue >>- model_resolution 2*time.Hour >>- model_frequency 3*time.Hour >> >> I highly appreciate your suggestions regarding this. >> >> Thank you. >> >> On Fri, Apr 8, 2016 at 5:05 PM, Malmee Weerasinghe >> wrote: >> >>> Hi All, >>> I have been working on detecting the idle containers in appcloud. I have >>> installed Heapster on Kubernetes and Heapster Metric Model gives only up to >>> 15 minutes of historical data for any Container. >>> >>> I have tried adding the flag "-model_resolution" and making it more >>> than 30s, but the flag can not be set as it gives errors when installing >>> the replication controller of heapster. >>> >>> Then I have tried adding "start=X&end=Y" to the end of the API >>> endpoint, but it gives the same response that gives without adding that >>> part. >>> >>> AFAIU if the database of heapster, influxdb stores all the data from 30s >>> to 30s, it needs a huge storage. Therefore it stores only up to 15 minutes >>> of historical data at any moment. >>> >>> To find the idle containers of appcloud, the resource usage details of >>> the containers have to be stored for a longer period than 15 minutes. One >>> of the approaches would be to use another database and store the resource >>> usage details of containers checking whether the container has been idle >>> for a period of 5 days. >>> >>> I highly appreciate your suggestions on the approaches that can be taken >>> regarding this. >>> >>> Thank you. >>> >>> -- >>> Malmee Weerasinghe >>> WSO2 Intern >>> mobile : (+94)* 71 7601905* | email : >>> mal...@wso2.com >>> >> >> >> >> -- >> Malmee Weerasinghe >> WSO2 Intern >> mobile : (+94)* 71 7601905* | email : >> mal...@wso2.com >> > > > > -- > *Imesh Gunaratne* > Senior Technical Lead > WSO2 Inc: http://wso2.com > T: +94 11 214 5345 M: +94 77 374 2057 > W: http://imesh.io > Lean . Enterprise . Middleware > > -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [AppCloud] Detecting idle containers in appcloud
Hi All, As there is a default database 'InfluxDB' that comes with 'Heapster' using the same database would be a better approach than an external database. Default Heapster gives only up to 15 minutes of historical data for any Container. Would using a custom Heapster docker image be a better approach on this? In Heapster Repository, Heapster.go file contains these configurations. - stats_resolution (Deafult 1*time.Second) The resolution at which heapster will retain stats - sink_frequency (Default 10*time.Second) Frequency at which data will be pushed to sinks - cache_duration (Default 4*time.Minute) The total duration of the historical data that will be cached by heapster - use_model (Default true) When true, the internal model representation will be used - model_resolution (Default 1*time.Minute) The resolution of the time series stored in the model. - model_frequency (Default 145*time.Second) Frequency at which model will be updated. * 1 second < stats_resolution < model_resolution < model_frequency sink_frequency < cache_duration* If these configuration are changed as below, build a custom docker image and use that docker image instead of default Heapster image Heapster:canary, would that be a better approach on this? Or is there any other approaches for this? - stats_resolution1*time.Hour - sink_frequency 2*time.Hour - cache_duration 3*time.Hour - use_modeltrue - model_resolution 2*time.Hour - model_frequency 3*time.Hour I highly appreciate your suggestions regarding this. Thank you. On Fri, Apr 8, 2016 at 5:05 PM, Malmee Weerasinghe wrote: > Hi All, > I have been working on detecting the idle containers in appcloud. I have > installed Heapster on Kubernetes and Heapster Metric Model gives only up to > 15 minutes of historical data for any Container. > > I have tried adding the flag "-model_resolution" and making it more than > 30s, but the flag can not be set as it gives errors when installing the > replication controller of heapster. > > Then I have tried adding "start=X&end=Y" to the end of the API endpoint, > but it gives the same response that gives without adding that part. > > AFAIU if the database of heapster, influxdb stores all the data from 30s > to 30s, it needs a huge storage. Therefore it stores only up to 15 minutes > of historical data at any moment. > > To find the idle containers of appcloud, the resource usage details of the > containers have to be stored for a longer period than 15 minutes. One of > the approaches would be to use another database and store the resource > usage details of containers checking whether the container has been idle > for a period of 5 days. > > I highly appreciate your suggestions on the approaches that can be taken > regarding this. > > Thank you. > > -- > Malmee Weerasinghe > WSO2 Intern > mobile : (+94)* 71 7601905* | email : > mal...@wso2.com > -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [AppCloud] Detecting idle containers in appcloud
Hi All, I have been working on detecting the idle containers in appcloud. I have installed Heapster on Kubernetes and Heapster Metric Model gives only up to 15 minutes of historical data for any Container. I have tried adding the flag "-model_resolution" and making it more than 30s, but the flag can not be set as it gives errors when installing the replication controller of heapster. Then I have tried adding "start=X&end=Y" to the end of the API endpoint, but it gives the same response that gives without adding that part. AFAIU if the database of heapster, influxdb stores all the data from 30s to 30s, it needs a huge storage. Therefore it stores only up to 15 minutes of historical data at any moment. To find the idle containers of appcloud, the resource usage details of the containers have to be stored for a longer period than 15 minutes. One of the approaches would be to use another database and store the resource usage details of containers checking whether the container has been idle for a period of 5 days. I highly appreciate your suggestions on the approaches that can be taken regarding this. Thank you. -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Resource level monitoring dashboard for applications on AppCloud
Hi All, When I tried as mentioned in documentation [1] to access cluster metrics (resource usage) through the REST endpoints it gives null values. But the data is passed to the storage backend, "InfluxDB" when I checked the data tables in the database. AFAIU there are missed configurations in the "Heapster" that I currently use, which comes in default in Kubernetes. As suggested by Manjula, the current approach is to delete the "Heapster" pod and re- create the pod with correct configurations. I would highly appreciate your suggestions on this approach. [1] https://github.com/kubernetes/heapster/blob/master/docs/model.md Thank you. On Mon, Mar 14, 2016 at 11:24 AM, Malmee Weerasinghe wrote: > Hi All, > I am working on building a dashboard for resource level monitoring of the > pods in kubernetes used by AppCloud. > > In kubernetes this is the structure used for resource usage monitoring. > cAdvisor > auto-discovers all containers in a node and collects CPU, memory, > filesystem, and network usage statistics while Heapster queries all these > data from the nodes through Kubelet and Kubelet gives the data fetched from > the cAdvisor. > > I have studied on connecting to the cAdvisor and the cAdvisor UI which > gives all the resource usage data of containers in a node graphically. But > accessing these data through Heapster would be a better approach as > suggested by Manjula than accessing a node seperately. > > I am studing on how to access these data through Heapster and how to pump > the data received from cAdvisor API to the dashboard that is to be > implemented. > > This is the approach that I will be taking and would highly appreciate > your suggestions on this. > > > > Thank you > -- > Malmee Weerasinghe > WSO2 Intern > mobile : (+94)* 71 7601905* | email : > mal...@wso2.com > -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] Resource level monitoring dashboard for applications on AppCloud
Hi All, I am working on building a dashboard for resource level monitoring of the pods in kubernetes used by AppCloud. In kubernetes this is the structure used for resource usage monitoring. cAdvisor auto-discovers all containers in a node and collects CPU, memory, filesystem, and network usage statistics while Heapster queries all these data from the nodes through Kubelet and Kubelet gives the data fetched from the cAdvisor. I have studied on connecting to the cAdvisor and the cAdvisor UI which gives all the resource usage data of containers in a node graphically. But accessing these data through Heapster would be a better approach as suggested by Manjula than accessing a node seperately. I am studing on how to access these data through Heapster and how to pump the data received from cAdvisor API to the dashboard that is to be implemented. This is the approach that I will be taking and would highly appreciate your suggestions on this. Thank you -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] Clustering in Cloud Foundry
Hi All, AFAIU there are two methods of implementing clustering on cloud foundry. 1) *Lattice*- Lattice is an open source project for running containerized workloads on a cluster. It combines the container scheduling (container processes are balanced across the infrastructure, even as the infrastructure changes), per application dynamic routing and log aggregation components from Cloud Foundry as a stand-alone package for deploying and operating clustered applications. 2) *Hazelcast* - 1.Hazelcast as User Provided Service Creating a Hazelcast cluster of 2 or more members and ensure public access to the instances and then registering the cluster as a service. 2.Hazelcast in Embedded mode Hazelcast runs inside an application and when the application is pushed onto Cloud Foundry, it’s the application’s job to discover other cluster members. Discovery SPI allows users to define their own discovery strategy, that enables Hazelcast members to register themselves at Consul server and retrieve URLs of other cluster members to make connection. 3.Hazelcast as a Cloud Foundry Service Creating a service instance using service broker and then create a cluster using the service. This process of research is done for implementing a carbon membership scheme for Cloud Foundry and your inputs are highly appreciated on this regard. Thank you -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] Running WSO2 Products on Cloud Foundry
Hi All, As an initiative for the $subject I have deployed Cloud Foundry with BOSH Lite, have run some sample applications both in BOSH lite and Pivotal Cloud Foundry. BOSH Lite is a pre-built Vagrant box which includes the BOSH Director. BOSH Lite is used for developing and testing releases locally. Cloud Foundry is now owned by Pivotal software and Pivotal Cloud Foundry is a commercial implementation. I have written two blog posts as an introduction to Cloud Foundry [1] and Bosh and Bosh Lite in Cloud Foundry [2]. As an approach to the $subject I am deploying tomcat server in Cloud Foundry and am expecting to replace the deployment with a WSO2 product (docker). Your suggestions on this subject are highly appreciated. [1] http://malmeeweerasinghe.blogspot.com/2016/02/introduction-to-cloud-foundry.html <http://malmeeweerasinghe.blogspot.com/2016/02/bosh-and-bosh-lite-in-cloud-foundry.html> [2] http://malmeeweerasinghe.blogspot.com/2016/02/bosh-and-bosh-lite-in-cloud-foundry.html Thanks -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Running WSO2 Products on Cloud Foundry
Hi Imesh, I have done a background research on Cloud Foundry and the blog under the following link contains the documentation of the research. I will include more posts on deploying Cloud Foundry with bosh lite and running simple applications on it, as I have studied so far. http://malmeeweerasinghe.blogspot.com/2016/02/introduction-to-cloud-foundry.html Thanks On Fri, Feb 5, 2016 at 10:31 PM, Imesh Gunaratne wrote: > Hi All, > > Shall we summarize the research work we have done so far and our approach > towards $subject? > > I know some of us have already done below, it might be better to document > this: > >- PaaS features of CF >- Steps for setting up a local environment with Vagrant >- Running a hello world sample >- Running a JVM using a standalone framework > > Next we might need to check following: > > 1. The ability to use Docker on CF > 2. The process of creating and managing Warden container images > 3. Find a mechanism to discover the member list of a cluster and implement > a Carbon membership scheme > 4. Create standalone frameworks for Carbon products > 5. Find a way to apply patches and software updates > 6. Find a way to implement a centralized logging solution > 7. Check whether there is a way to monitor the health of the containers > (similar to cAdvisor and Cockpit UI in Kubernetes) > 8. Create artifacts required for deploying a Carbon server on CF and > prepare a guideline. > > [1] > https://github.com/wso2/kubernetes-artifacts/tree/master/common/kubernetes-membership-scheme > > > Thanks > > -- > *Imesh Gunaratne* > Senior Technical Lead > WSO2 Inc: http://wso2.com > T: +94 11 214 5345 M: +94 77 374 2057 > W: http://imesh.io > Lean . Enterprise . Middleware > > -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [VOTE] Release WSO2 Puppet Modules 1.0.0 RC2
;>>>>> > problems. shall we fix that. >>>>>>>> > regards >>>>>>>> > Ramith >>>>>>>> > >>>>>>>> > On Mon, Feb 1, 2016 at 11:14 AM, Isuru Haththotuwa < >>>>>>>> isu...@wso2.com> wrote: >>>>>>>> >> Hi devs, >>>>>>>> >> >>>>>>>> >> This is the second release candidate of WSO2 Puppet Modules >>>>>>>> 1.0.0. Please >>>>>>>> >> download, test and vote. >>>>>>>> >> >>>>>>>> >> [ ] Broken - do not release (explain why) >>>>>>>> >> [ ] Stable - go ahead and release >>>>>>>> >> >>>>>>>> >> The tag to be voted upon: >>>>>>>> >> https://github.com/wso2/puppet-modules/tree/v1.0.0-rc2 >>>>>>>> >> >>>>>>>> >> Source and binary distribution files: >>>>>>>> >> https://github.com/wso2/puppet-modules/releases/tag/v1.0.0-rc2 >>>>>>>> >> >>>>>>>> >> The vote will be open for 72 hours or as needed. >>>>>>>> >> >>>>>>>> >> -- >>>>>>>> >> Thanks, >>>>>>>> >> The WSO2 Private PaaS Team >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> ___ >>>>>>>> >> Dev mailing list >>>>>>>> >> Dev@wso2.org >>>>>>>> >> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>> >> >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > -- >>>>>>>> > Ramith Jayasinghe >>>>>>>> > Technical Lead >>>>>>>> > WSO2 Inc., http://wso2.com >>>>>>>> > lean.enterprise.middleware >>>>>>>> > >>>>>>>> > E: ram...@wso2.com >>>>>>>> > P: +94 777542851 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Ramith Jayasinghe >>>>>>>> Technical Lead >>>>>>>> WSO2 Inc., http://wso2.com >>>>>>>> lean.enterprise.middleware >>>>>>>> >>>>>>>> E: ram...@wso2.com >>>>>>>> P: +94 777542851 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thanks and Regards, >>>>>>> >>>>>>> Isuru H. >>>>>>> +94 716 358 048* <http://wso2.com/>* >>>>>>> >>>>>>> >>>>>>> >>>>>>> ___ >>>>>>> Dev mailing list >>>>>>> Dev@wso2.org >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> ___ >>>>>> Dev mailing list >>>>>> Dev@wso2.org >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Thanuja Uruththirakodeeswaran >>>>> Software Engineer >>>>> WSO2 Inc.;http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> mobile: +94 774363167 >>>>> >>>>> ___ >>>>> Dev mailing list >>>>> Dev@wso2.org >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Vishanth Balasubramaniam* >>>> Committer & PMC Member, Apache Stratos, >>>> Software Engineer, WSO2 Inc.; http://wso2.com >>>> >>>> mobile: *+94 77 17 377 18* >>>> about me: *http://about.me/vishanth <http://about.me/vishanth>* >>>> >>>> ___ >>>> Dev mailing list >>>> Dev@wso2.org >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> *Pubudu Gunatilaka* >>> Committer and PMC Member - Apache Stratos >>> Software Engineer >>> WSO2, Inc.: http://wso2.com >>> mobile : +94774079049 <%2B94772207163> >>> >>> >>> ___ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> Sajith Kariyawasam >> *Committer and PMC member, Apache Stratos, * >> *WSO2 Inc.; http://wso2.com <http://wso2.com>* >> *Mobile: 0772269575* >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Nishadi Kirielle > *Software Engineering Intern* > Mobile : +94 (0) 714722148 > nish...@wso2.com > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] PPaaS artifact conversion tool- Creating multi tenant PPaaS applications
Hi All, We are currently implementing a tool to convert artifact json files of PPaaS 4.0.0 to PPaaS 4.1.x. In the implemented tool we have considered only the single tenant cartridges. We are improving the tool to convert multi tenant cartridges to multi tenant applications. The approach we are taking is to get all the multi tenant cartridges and all the deployed services and map a service for each cartridge. In creating multi tenant applications, following mapping has been used. Would it be accurate to use service name as the application alias? Your suggestions on this conversion are highly appreciated. Thanks -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] PPaaS Artifact Migration Tool- Naming Convention for Application ID
Hi All, In the tool that is being developed to convert artifact JSON files of PPaaS 4.0.0 to PPaaS 4.1.x, the conversion has to be done as such one application has to be created for one subscription. In this conversion what should be the naming convention of the application ID? One approach is to use 'alias' in subscription data as an ID for a particular application. Thanks -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Test cases for Artifact Migration Tool PPaaS 4.0.0 to 4.1.0
Hi Imesh, I have used servlets to implement the REST API as I am more familiar with servlets than JAX-RS.Is it fine to use servlets or should I use JAX-RS? In here two servlets are used as the paths used are different when retrieving the JSON files. In one servlet the context path is migration/admin while in the other the context path is stratos/admin. The values for the constants are different in the test cases. The paths used are different in the test cases and therefore the values are assigned in test cases. Thanks On Fri, Jan 8, 2016 at 1:30 PM, Imesh Gunaratne wrote: > Hi Malmee, > > I have few questions: > >- Why do we use servlets to implement a REST API instead of JAX-RS? >Have a look at [1]. >- Why are we having two servlets instead of one? >- Why do we assign values to constants in runtime, don't they need to >be final? > > Constants.CERTIFICATE_PATH = TestConstants.TEST_CERTIFICATE; > Constants.DIRECTORY_SOURCE_SCRIPT = > TestConstants.TEST_DIRECTORY_SOURCE_SCRIPT; > Constants.ROOT_DIRECTORY = TestConstants.OUTPUT_DIRECTORY; > > > [1] > https://nikolaygrozev.wordpress.com/2014/10/16/rest-with-embedded-jetty-and-jersey-in-a-single-jar-step-by-step/ > > Thanks > > On Fri, Jan 8, 2016 at 9:38 AM, Malmee Weerasinghe > wrote: > >> Hi All, >> We have added unit tests for the Artifact Migration Tool which converts >> PPaaS 4.0.0 artifact JSON files to PPaaS 4.1.x. [1] >> >> The testing is done using Jetty web server to mock the REST API by adding >> two servlets. The response JSONs are transformed from the tool and the >> output files are tested with a set of sample output files in the test case. >> >> [1] >> https://github.com/nishadi/product-private-paas/tree/master/tools/migration/ppaas-artifact-converter >> >> Thanks >> >> >> -- >> Malmee Weerasinghe >> WSO2 Intern >> mobile : (+94)* 71 7601905* | email : >> mal...@wso2.com >> > > > > -- > *Imesh Gunaratne* > Senior Technical Lead > WSO2 Inc: http://wso2.com > T: +94 11 214 5345 M: +94 77 374 2057 > W: http://imesh.gunaratne.org > Lean . Enterprise . Middleware > > -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] Test cases for Artifact Migration Tool PPaaS 4.0.0 to 4.1.0
Hi All, We have added unit tests for the Artifact Migration Tool which converts PPaaS 4.0.0 artifact JSON files to PPaaS 4.1.x. [1] The testing is done using Jetty web server to mock the REST API by adding two servlets. The response JSONs are transformed from the tool and the output files are tested with a set of sample output files in the test case. [1] https://github.com/nishadi/product-private-paas/tree/master/tools/migration/ppaas-artifact-converter Thanks -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] HTTPS REST Client
Hi All,
The method I have built doesn't need any dependencies. That was the main
concern for building it this way. But as Kasun suggests I will use existing
libraries to build the rest client considering the need of more
functionality.
Thank you
On Fri, Dec 18, 2015 at 12:25 PM, Kasun Indrasiri wrote:
> Any reason to building a HTTP client from scratch? Why don't we use
> existing libs such as HTTPClient or Netty?
>
> On Fri, Dec 18, 2015 at 8:51 AM, Harsha Thirimanna
> wrote:
>
>> Just try to give absolute path first and test. Because as Dharshana
>> mentioned, it should work.
>>
>>
>> *Harsha Thirimanna*
>> Senior Software Engineer; WSO2, Inc.; http://wso2.com
>> * <http://www.apache.org/>*
>> *email: **hars...@wso2.com* * cell: +94 71 5186770 *
>> *twitter: **http://twitter.com/ <http://twitter.com/afkham_azeez>*
>> *harshathirimannlinked-in: **http:
>> <http://lk.linkedin.com/in/afkhamazeez>**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122
>> <http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122>*
>>
>> *Lean . Enterprise . Middleware*
>>
>>
>> On Thu, Dec 17, 2015 at 5:23 PM, Dharshana Warusavitharana <
>> dharsha...@wso2.com> wrote:
>>
>>> Hi Malmee,
>>>
>>> You have to replace the cert location with actuvel .srt file you have.
>>>
>>> Thank you,
>>> Dharshana
>>>
>>> On Thu, Dec 17, 2015 at 2:32 PM, Malmee Weerasinghe
>>> wrote:
>>>
>>>> Hi Dharshana,
>>>> I tried with what you suggested. But I still get the same error.
>>>> ppaas-artifact-converter tool doesn't run on Carbon and it runs
>>>> independently. Therefore for "System.getProperty("carbon.home")" it
>>>> returns null. I tried copying this file, "wso2carbon.jks" to the resources
>>>> folder. But I get the same error.
>>>> Thank you.
>>>>
>>>> On Thu, Dec 17, 2015 at 9:31 AM, Dharshana Warusavitharana <
>>>> dharsha...@wso2.com> wrote:
>>>>
>>>>> Hi Malmee,
>>>>>
>>>>> The SSL issue comes with the rest client is because you are going to
>>>>> invoke HTTPS backend with out proper certificate in path.
>>>>>
>>>>> you can set keys to the system path using following
>>>>>
>>>>> String trustStore = System.getProperty("carbon.home") + File.separator +
>>>>> "repository" + File.separator +
>>>>> "resources" + File.separator + "security" +
>>>>> File.separator + "wso2carbon.jks";
>>>>> System.setProperty("javax.net.ssl.trustStore", trustStore);
>>>>> System.setProperty("javax.net.ssl.trustStorePassword", "wso2carbon");
>>>>> System.setProperty("javax.net.ssl.trustStoreType", "JKS");
>>>>>
>>>>>
>>>>> In this sample WSO2 carbon keys set as keys to the system path you can
>>>>> use the same.
>>>>>
>>>>> Apache HTTP client is bit old client instead you can try some thing like
>>>>> JAX-RS 2 or RestEasy client.
>>>>>
>>>>> In All these clients you need to add above code segment to export keys to
>>>>> call secured HTTP call.
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Dharshana.
>>>>>
>>>>>
>>>>> On Wed, Dec 16, 2015 at 8:56 PM, Isuru Haththotuwa
>>>>> wrote:
>>>>>
>>>>>> Hi Malmee,
>>>>>>
>>>>>> If you have used java built in methods, that would mean your tool
>>>>>> would not be required to depend on an external library such as Apache
>>>>>> HTTP
>>>>>> client as you have mentioned. This is fine for a simple use case.
>>>>>> However,
>>>>>> please note if you need more functionality, such as support for all HTTP
>>>>>> operations, and other capabilities, etc. it would be advisable to use
>>>>>> Apache HTTP client or any other suitable existing library without
>>>>>> implementing it again by yourself.
>>>>>>
>>>>>> For the certificate issue that you are getting, its possible to
>>>>>> override the default certificate validation mechanis
Re: [Dev] HTTPS REST Client
Hi All,
I have tried with the actual path and I get the same error. AFAIU this have
to be a certificate error. [1]
As Dharshana suggested I would use existing libraries to build the HTTPS
client.
[1] https://github.com/nishadi/product-private-paas/pull/2
Thank you
On Fri, Dec 18, 2015 at 12:25 PM, Kasun Indrasiri wrote:
> Any reason to building a HTTP client from scratch? Why don't we use
> existing libs such as HTTPClient or Netty?
>
> On Fri, Dec 18, 2015 at 8:51 AM, Harsha Thirimanna
> wrote:
>
>> Just try to give absolute path first and test. Because as Dharshana
>> mentioned, it should work.
>>
>>
>> *Harsha Thirimanna*
>> Senior Software Engineer; WSO2, Inc.; http://wso2.com
>> * <http://www.apache.org/>*
>> *email: **hars...@wso2.com* * cell: +94 71 5186770 *
>> *twitter: **http://twitter.com/ <http://twitter.com/afkham_azeez>*
>> *harshathirimannlinked-in: **http:
>> <http://lk.linkedin.com/in/afkhamazeez>**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122
>> <http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122>*
>>
>> *Lean . Enterprise . Middleware*
>>
>>
>> On Thu, Dec 17, 2015 at 5:23 PM, Dharshana Warusavitharana <
>> dharsha...@wso2.com> wrote:
>>
>>> Hi Malmee,
>>>
>>> You have to replace the cert location with actuvel .srt file you have.
>>>
>>> Thank you,
>>> Dharshana
>>>
>>> On Thu, Dec 17, 2015 at 2:32 PM, Malmee Weerasinghe
>>> wrote:
>>>
>>>> Hi Dharshana,
>>>> I tried with what you suggested. But I still get the same error.
>>>> ppaas-artifact-converter tool doesn't run on Carbon and it runs
>>>> independently. Therefore for "System.getProperty("carbon.home")" it
>>>> returns null. I tried copying this file, "wso2carbon.jks" to the resources
>>>> folder. But I get the same error.
>>>> Thank you.
>>>>
>>>> On Thu, Dec 17, 2015 at 9:31 AM, Dharshana Warusavitharana <
>>>> dharsha...@wso2.com> wrote:
>>>>
>>>>> Hi Malmee,
>>>>>
>>>>> The SSL issue comes with the rest client is because you are going to
>>>>> invoke HTTPS backend with out proper certificate in path.
>>>>>
>>>>> you can set keys to the system path using following
>>>>>
>>>>> String trustStore = System.getProperty("carbon.home") + File.separator +
>>>>> "repository" + File.separator +
>>>>> "resources" + File.separator + "security" +
>>>>> File.separator + "wso2carbon.jks";
>>>>> System.setProperty("javax.net.ssl.trustStore", trustStore);
>>>>> System.setProperty("javax.net.ssl.trustStorePassword", "wso2carbon");
>>>>> System.setProperty("javax.net.ssl.trustStoreType", "JKS");
>>>>>
>>>>>
>>>>> In this sample WSO2 carbon keys set as keys to the system path you can
>>>>> use the same.
>>>>>
>>>>> Apache HTTP client is bit old client instead you can try some thing like
>>>>> JAX-RS 2 or RestEasy client.
>>>>>
>>>>> In All these clients you need to add above code segment to export keys to
>>>>> call secured HTTP call.
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Dharshana.
>>>>>
>>>>>
>>>>> On Wed, Dec 16, 2015 at 8:56 PM, Isuru Haththotuwa
>>>>> wrote:
>>>>>
>>>>>> Hi Malmee,
>>>>>>
>>>>>> If you have used java built in methods, that would mean your tool
>>>>>> would not be required to depend on an external library such as Apache
>>>>>> HTTP
>>>>>> client as you have mentioned. This is fine for a simple use case.
>>>>>> However,
>>>>>> please note if you need more functionality, such as support for all HTTP
>>>>>> operations, and other capabilities, etc. it would be advisable to use
>>>>>> Apache HTTP client or any other suitable existing library without
>>>>>> implementing it again by yourself.
>>>>>>
>>>>>> For the certificate issue that you are getting, its possible to
>>>>>> override the default certifica
Re: [Dev] HTTPS REST Client
Hi Dharshana,
I tried with what you suggested. But I still get the same error.
ppaas-artifact-converter tool doesn't run on Carbon and it runs
independently. Therefore for "System.getProperty("carbon.home")" it returns
null. I tried copying this file, "wso2carbon.jks" to the resources folder.
But I get the same error.
Thank you.
On Thu, Dec 17, 2015 at 9:31 AM, Dharshana Warusavitharana <
dharsha...@wso2.com> wrote:
> Hi Malmee,
>
> The SSL issue comes with the rest client is because you are going to
> invoke HTTPS backend with out proper certificate in path.
>
> you can set keys to the system path using following
>
> String trustStore = System.getProperty("carbon.home") + File.separator +
> "repository" + File.separator +
> "resources" + File.separator + "security" +
> File.separator + "wso2carbon.jks";
> System.setProperty("javax.net.ssl.trustStore", trustStore);
> System.setProperty("javax.net.ssl.trustStorePassword", "wso2carbon");
> System.setProperty("javax.net.ssl.trustStoreType", "JKS");
>
>
> In this sample WSO2 carbon keys set as keys to the system path you can use
> the same.
>
> Apache HTTP client is bit old client instead you can try some thing like
> JAX-RS 2 or RestEasy client.
>
> In All these clients you need to add above code segment to export keys to
> call secured HTTP call.
>
>
> Thank you,
>
> Dharshana.
>
>
> On Wed, Dec 16, 2015 at 8:56 PM, Isuru Haththotuwa
> wrote:
>
>> Hi Malmee,
>>
>> If you have used java built in methods, that would mean your tool would
>> not be required to depend on an external library such as Apache HTTP client
>> as you have mentioned. This is fine for a simple use case. However, please
>> note if you need more functionality, such as support for all HTTP
>> operations, and other capabilities, etc. it would be advisable to use
>> Apache HTTP client or any other suitable existing library without
>> implementing it again by yourself.
>>
>> For the certificate issue that you are getting, its possible to override
>> the default certificate validation mechanism and plug in your own
>> implementation which can disable certificate validation (for testing
>> purposes). For HTTP Client 4, please see [1].
>>
>> [1].
>> http://stackoverflow.com/questions/2703161/how-to-ignore-ssl-certificate-errors-in-apache-httpclient-4-0
>>
>> On Wed, Dec 16, 2015 at 8:29 PM, Malmee Weerasinghe
>> wrote:
>>
>>> Hi All,
>>>
>>> We have developed a HTTPS REST client using java in-built methods which
>>> works properly. [1] This method is configured to allow self signed
>>> certificates.
>>>
>>> When using apache http client we get a certificate error :
>>> javax.net.ssl.SSLPeerUnverifiedException: Host name '192.168.30.227'
>>> does not match the certificate subject provided by the peer (CN=localhost,
>>> O=WSO2, L=Mountain View, ST=CA, C=US).
>>>
>>> What would be a good choice? Using apache HTTP client or java in built
>>> methods. Your suggestions are highly appreciated.
>>>
>>> [1]
>>> https://github.com/nishadi/product-private-paas/blob/master/tools/migration/ppaas-artifact-converter/src/main/java/org/wso2/ppaas/tools/artifactmigration/RestClient.java
>>>
>>> --
>>> Malmee Weerasinghe
>>> WSO2 Intern
>>> mobile : (+94)* 71 7601905* | email :
>>> mal...@wso2.com
>>>
>>> ___
>>> Dev mailing list
>>> Dev@wso2.org
>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>
>>>
>>
>>
>> --
>> Thanks and Regards,
>>
>> Isuru H.
>> +94 716 358 048* <http://wso2.com/>*
>>
>>
>>
>> ___
>> Dev mailing list
>> Dev@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
>
> --
>
> Dharshana Warusavitharana
> Senior Software Engineer , Test Automation
> WSO2 Inc. http://wso2.com
> email : dharsha...@wso2.com
> Tel : +94 11 214 5345
> Fax :+94 11 2145300
> cell : +94770342233
> blog : http://dharshanaw.blogspot.com
>
> lean . enterprise . middleware
>
--
Malmee Weerasinghe
WSO2 Intern
mobile : (+94)* 71 7601905* | email :
mal...@wso2.com
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] HTTPS REST Client
Hi All, We have developed a HTTPS REST client using java in-built methods which works properly. [1] This method is configured to allow self signed certificates. When using apache http client we get a certificate error : javax.net.ssl.SSLPeerUnverifiedException: Host name '192.168.30.227' does not match the certificate subject provided by the peer (CN=localhost, O=WSO2, L=Mountain View, ST=CA, C=US). What would be a good choice? Using apache HTTP client or java in built methods. Your suggestions are highly appreciated. [1] https://github.com/nishadi/product-private-paas/blob/master/tools/migration/ppaas-artifact-converter/src/main/java/org/wso2/ppaas/tools/artifactmigration/RestClient.java -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Locating the Artifact Converter Tool in WSO2 Product-Private-PaaS Repo
@Akila: No, we haven't implemented that feature, to convert cartridge subscription artifacts to application sign ups and domain mapping subscriptions. We still need to check the tool on an actual PPaaS 4.0.0 before locating it in the repo as we implemented it by mocking the end points. Will send a PR once we are done. Thanks On Tue, Dec 1, 2015 at 8:54 PM, Imesh Gunaratne wrote: > > On Tue, Dec 1, 2015 at 7:38 PM, Gayan Gunarathne wrote: > >> >> On Tue, Dec 1, 2015 at 6:46 PM, Akila Ravihansa Perera < >> raviha...@wso2.com> wrote: >> >>> Hi, >>> >>> +1 for the proposed folder structure. >>> >>> @Gayan: Currently that tool only exports existing cartridge >>> subscriptions and domain mappings. It doesn't do any conversion or >>> migration (although it is called migration-tool); >>> >> >> In that case, if we put this under tools/migration it will be misleading >> for the end user. >> > > Yes, that's why we need to rename it. +1 for calling it > subscription-exporter. > > Thanks > >> >> >>> which is why it should be renamed to subscription-manager. I actually >>> prefer the name "subscription-exporter". We had to create an external tool >>> since regular API didn't have any methods to expose those information. We >>> should include those API methods to the regular API in the new PPaaS >>> release. >>> >>> @Malmee: I've restructured the folder structure in [1]. You can create a >>> new folder named "artifact-converter" at tools/migration/ to host your >>> tool. Please send a PR with those changes. >>> >>> On a side note; does your tool support converting cartridge subscription >>> artifacts to application signups and domain mapping subscriptions? >>> >>> [1] >>> https://github.com/wso2/product-private-paas/tree/master/tools/migration >>> >>> Thanks. >>> >>> On Tue, Dec 1, 2015 at 6:23 PM, Imesh Gunaratne wrote: >>> >>>> >>>> >>>> On Tue, Dec 1, 2015 at 5:23 PM, Gayan Gunarathne >>>> wrote: >>>> >>>>> >>>>> On Tue, Dec 1, 2015 at 4:46 PM, Imesh Gunaratne >>>>> wrote: >>>>> >>>>>> Shall we have something like below: >>>>>> >>>>>> └── tools >>>>>> └── migration >>>>>> ├── artifact-converter >>>>>> └── subcription-manager >>>>>> >>>>> >>>>> I think it is a subscription-converter not subscription-manager.It >>>>> will convert the 4.0.0 subscriptions to the 4.1.0. So shall we call it as >>>>> subscription-converter? >>>>> >>>> >>>> AFAIK it does not convert subscriptions, it just export them. >>>> >>>> Thanks >>>> >>>>> >>>>>> @Akila: We might need to rename the existing subscription management >>>>>> tool. >>>>>> >>>>>> Thanks >>>>>> >>>>>> On Tue, Dec 1, 2015 at 3:37 PM, Malmee Weerasinghe >>>>>> wrote: >>>>>> >>>>>>> Hi Akila, >>>>>>> We need to locate the Artifact Converter Tool which converts PPaaS >>>>>>> 4.0.0 artifacts to PPaaS 4.1.0, in Product-Private-PaaS Repo. >>>>>>> >>>>>>> As Artifact Converter Tool and paas-migration/4.0.0 tool have quite >>>>>>> similar functionality, can we create a new folder in 'tools' and move >>>>>>> paas-migration/4.0.0 tool to it and locate together with Artifact >>>>>>> Converter Tool. Do you have any suggestions? >>>>>>> >>>>>>> Thank you >>>>>>> -- >>>>>>> Malmee Weerasinghe >>>>>>> WSO2 Intern >>>>>>> mobile : (+94)* 71 7601905* | email : >>>>>>> mal...@wso2.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Imesh Gunaratne* >>>>>> Senior Technical Lead >>>>>> WSO2 Inc: http://wso2.com >>>>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>>>> W: http://imesh.gunaratne.org >>>>>> Lean . Enterprise . Middleware &
[Dev] Locating the Artifact Converter Tool in WSO2 Product-Private-PaaS Repo
Hi Akila, We need to locate the Artifact Converter Tool which converts PPaaS 4.0.0 artifacts to PPaaS 4.1.0, in Product-Private-PaaS Repo. As Artifact Converter Tool and paas-migration/4.0.0 tool have quite similar functionality, can we create a new folder in 'tools' and move paas-migration/4.0.0 tool to it and locate together with Artifact Converter Tool. Do you have any suggestions? Thank you -- Malmee Weerasinghe WSO2 Intern mobile : (+94)* 71 7601905* | email : mal...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
