Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-24 Thread Muhammed Shariq
On Tue, Jan 24, 2012 at 6:51 PM, Krishantha Samaraweera wrote: > Hi Shariq, > > IMO, we can think about Stratos deployment based on trunk builds after we > finish with Stratos Services Deployment management tool. > Yes +1. I had a chat with Chamith and Sanjaya regarding how we can setup the pack

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-24 Thread Krishantha Samaraweera
Hi Shariq, IMO, we can think about Stratos deployment based on trunk builds after we finish with Stratos Services Deployment management tool. At the moment what we can easily do is to, setting up the framework to monitor SL. We can configure it to run once per every two days. WDYT? Thanks, Krish

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-24 Thread Charitha Kankanamge
We will have a separate Gdoc to capture the platform test scenarios. See [1] (in progress version) The test artifacts produced by us in these scenarios will be merged into the system framework. [1]https://docs.google.com/a/wso2.com/spreadsheet/ccc?key=0Arh4GwhkOX3ndEZnRll4eFhjRjhqVGlJUEd1b0FsZ0E&h

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-22 Thread Muhammed Shariq
On Fri, Jan 20, 2012 at 12:59 PM, Krishantha Samaraweera < krishan...@wso2.com> wrote: > > On Fri, Jan 20, 2012 at 9:51 AM, Muhammed Shariq wrote: > >> Hi Krishantha et al, >> >> If I am not mistaken we are running these test for products only right, >> at the moment? How about the possibility of

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-20 Thread Charitha Kankanamge
Hi Srinath, We already have TestPlan docs such as [1] which we maintain against each release. However we categorized them according to products. What we can do is, add an additional sheet with "platform test scenarios" and include the cases which needs to be done, which has already been automated

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Srinath Perera
Shall we have a test metric defined in a google doc and keep track of what to be done etc through that test matrix? --Srinath On Fri, Jan 20, 2012 at 1:00 PM, Charitha Kankanamge wrote: > > On Fri, Jan 20, 2012 at 11:44 AM, Krishantha Samaraweera > wrote: >> >> Hi Srinath, >> >> On Fri, Jan 20,

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Charitha Kankanamge
On Fri, Jan 20, 2012 at 11:44 AM, Krishantha Samaraweera < krishan...@wso2.com> wrote: > Hi Srinath, > > On Fri, Jan 20, 2012 at 5:28 AM, Srinath Perera wrote: > >> Thanks Krishantha!! >> >> How good are we with test coverage (including introps and perf)? Can >> you send the current status and pl

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Krishantha Samaraweera
On Fri, Jan 20, 2012 at 9:51 AM, Muhammed Shariq wrote: > Hi Krishantha et al, > > If I am not mistaken we are running these test for products only right, at > the moment? How about the possibility of running these test on the stratos > services too ? I had a chat with Dhrashana and we most of th

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Charitha Kankanamge
On Fri, Jan 20, 2012 at 9:51 AM, Muhammed Shariq wrote: > Hi Krishantha et al, > > If I am not mistaken we are running these test for products only right, at > the moment? How about the possibility of running these test on the stratos > services too ? I had a chat with Dhrashana and we most of th

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Afkham Azeez
Excellent thoughts Shariq. Integration tests for Stratos are essential. I guess what we should do is, do a full build, run the Stratos setup script, create tenants, run some tenant management tests, run some tenant specific tests, platform sample test etc. Please consult your MC on working with th

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Krishantha Samaraweera
Hi Srinath, On Fri, Jan 20, 2012 at 5:28 AM, Srinath Perera wrote: > Thanks Krishantha!! > > How good are we with test coverage (including introps and perf)? Can > you send the current status and plan to cover other products? > As I mentioned in the earlier mail, there were few issue with Ravan

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Muhammed Shariq
Hi Krishantha et al, If I am not mistaken we are running these test for products only right, at the moment? How about the possibility of running these test on the stratos services too ? I had a chat with Dhrashana and we most of the tests that covers multitenancy aspect, so its a matter of setting

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Srinath Perera
Thanks Krishantha!! How good are we with test coverage (including introps and perf)? Can you send the current status and plan to cover other products? Did we start running Bamboo again since this is fixed? Do we run introps and perf tests with this? --Srinath On Fri, Jan 20, 2012 at 12:28 AM,

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Krishantha Samaraweera
Framework users maven as the builder. Hence, it uses dependencies relevant to carbon version under test. Support for multiple carbon versions will be a good additions to the framework. When consider test architecture, having support for multiple carbon versions, will add extra complexity to the fra

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Tharindu Mathew
Hi Kris, I was wondering whether there is a possibility of testing different carbon versions of products, such as ESB 3.0.1 with BAM 2.0.0? On Fri, Jan 20, 2012 at 12:28 AM, Krishantha Samaraweera < krishan...@wso2.com> wrote: > We need to monitor how this works for few days. > If everything run

Re: [Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Krishantha Samaraweera
We need to monitor how this works for few days. If everything runs smoothly then we can enable test failure notifications. Thanks, Krishantha. On Fri, Jan 20, 2012 at 12:23 AM, Krishantha Samaraweera < krishan...@wso2.com> wrote: > Hi, > > We have done the $subject on Bamboo machine and it is wo

[Carbon-dev] System Test Framework Integration with Bamboo trunk build.

2012-01-19 Thread Krishantha Samaraweera
Hi, We have done the $subject on Bamboo machine and it is working. You can find the last successful test build at [1]. Test framework has been configured to use packs generated by trunk build. The test setup consists of five products (ESB, G-Reg, AS, BPS and DSS) with registry mounts on top of My