Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi, We have completed implementing this common class and we will commit this to GIT soon. Instead writing any java test class extending this common class we can simply add the test class to the testng.xml inside the [PRODUCT_HOME]/modules/integration/tests-integration/xxx/src/test/resources directory and it will get executed automatically. I have already tested this with EMM and it worked in the expected manner.Also note that due to some issues as mentioned in the thread '[Dev] Sugarcane release' we might not be able to run every test continuously till the mentioned bugs are fixed. eg: test name=Test - EMM preserve-order=true parallel=false classes class name=org.wso2.carbon.integration.common.tests.JaggeryServerTest/ /classes /test Regards On Thu, Jul 10, 2014 at 12:55 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Chamil On Thu, Jul 10, 2014 at 12:39 PM, Chamil Jeewantha cha...@wso2.com wrote: Hi All, I was thinking the tests are written in Java. I had a chat with Ruchira understood the tests are written in Jaggery. So let me take back the annotation. BTW, my personal opinion is, just putting a dummy java class to identify whether its Jaggery test is bit odd. Dummy class is not to identify the Jaggery tests, it does many more functionalities, like identify all endpoints in each Jaggery app and send a GET request to execute the tests represented at each endpoint. There are many other java based scripting languages which may have integrated with TestNG. I think its better to do a small research to find what is the way they have followed. [1] for example. WDYT? This is just a matter of sending GET request to Jaggery app endpoint. No scripting required IMO. Thanks, Krishantha. [1] https://code.google.com/p/testngroove/ On Thu, Jul 10, 2014 at 12:10 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Krishantha, If it is just a dummy class as you mentioned, then that would be fine. i.e. we neither need to maintain any *.xml configurations or Java codes, other than the pom.xml configuration. Regarding my initial suggestion, you would be able to do it using [1]. [1] http://testng.org/doc/documentation-main.html#running-testng-programmatically [2] http://stackoverflow.com/questions/13732015/dynamic-test-suites-using-testng On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi Krishantha, If it is just a dummy class as you mentioned, then that would be fine. i.e. we neither need to maintain any *.xml configurations or Java codes, other than the pom.xml configuration. Regarding my initial suggestion, you would be able to do it using [1]. [1] http://testng.org/doc/documentation-main.html#running-testng-programmatically [2] http://stackoverflow.com/questions/13732015/dynamic-test-suites-using-testng On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email: ruch...@wso2.com ruch...@wso2.com, blog: ruchirawageesha.blogspot.com http://ruchirawageesha.blogspot.com, mobile: +94 77 5493444 %2B94%2077%205493444* -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi All, I was thinking the tests are written in Java. I had a chat with Ruchira understood the tests are written in Jaggery. So let me take back the annotation. BTW, my personal opinion is, just putting a dummy java class to identify whether its Jaggery test is bit odd. There are many other java based scripting languages which may have integrated with TestNG. I think its better to do a small research to find what is the way they have followed. [1] for example. WDYT? [1] https://code.google.com/p/testngroove/ On Thu, Jul 10, 2014 at 12:10 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Krishantha, If it is just a dummy class as you mentioned, then that would be fine. i.e. we neither need to maintain any *.xml configurations or Java codes, other than the pom.xml configuration. Regarding my initial suggestion, you would be able to do it using [1]. [1] http://testng.org/doc/documentation-main.html#running-testng-programmatically [2] http://stackoverflow.com/questions/13732015/dynamic-test-suites-using-testng On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked.
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi Chamil, On Thu, Jul 10, 2014 at 10:57 AM, Chamil Jeewantha cha...@wso2.com wrote: AFAIU, If any test is extended from the base class, you identify it as a jaggery test and do some special treatment like integrating to Emma etc... The annotation marks the class as qualified to receive the special treatment you are going to do within the abstract class. The best example is the difference between JUnit 3 4. In JUnit 3 we had to extend our test class from junit.framework.TestCase class. But in Junit 4, the abstract class has been deprecated and introduced a simple annotations. So I think asking people to implement their test classes from a Abstract class is an old fashion which was followed before java 5. We can improve the readability of the test using annotations. Lets say we have set of tests which need to be executed with different configurations. Then we need to abstract the test methods and write test classes extending the abstract class to avoid repetition of tests. Ruchira's requirement is to code less test execution, so I don't think annotations would help here. We already use standard TestNG annotations, however implementing new annotation for each requirement does't going to scale. Thanks, Krishantha. On Thu, Jul 10, 2014 at 10:25 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Chamil, We can introduce TestNG custom annotation easily, but I'm in confusion how custom annotation fit in to this context. Only thing which comes to my mind is include the base class to all products using TestNG annotation transform listener. Thanks, Krishantha. On Thu, Jul 10, 2014 at 10:16 AM, Chamil Jeewantha cha...@wso2.com wrote: Hi All, Why can't we use an annotation like @JaggeryTest rather than a Base class in this case? It will be more sophisticated to the test writers. Regards, Chamil On Wed, Jul 9, 2014 at 11:59 AM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Krishantha, You should be able to use the same admin service that you use for Java webapps. On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7,
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi Ruchira, On Thu, Jul 10, 2014 at 12:10 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Krishantha, If it is just a dummy class as you mentioned, then that would be fine. i.e. we neither need to maintain any *.xml configurations or Java codes, other than the pom.xml configuration. Regarding my initial suggestion, you would be able to do it using [1]. [1] http://testng.org/doc/documentation-main.html#running-testng-programmatically [2] http://stackoverflow.com/questions/13732015/dynamic-test-suites-using-testng We used to run TestNG programatically in our early days. But it has many problems when it comes to test management (i.e you can't run selected tests without changing the test runner class) I'm researching around TestNG advance capabilities get the common test class running even without dummy class. Thanks, Krishantha. On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email :
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi Chamil On Thu, Jul 10, 2014 at 12:39 PM, Chamil Jeewantha cha...@wso2.com wrote: Hi All, I was thinking the tests are written in Java. I had a chat with Ruchira understood the tests are written in Jaggery. So let me take back the annotation. BTW, my personal opinion is, just putting a dummy java class to identify whether its Jaggery test is bit odd. Dummy class is not to identify the Jaggery tests, it does many more functionalities, like identify all endpoints in each Jaggery app and send a GET request to execute the tests represented at each endpoint. There are many other java based scripting languages which may have integrated with TestNG. I think its better to do a small research to find what is the way they have followed. [1] for example. WDYT? This is just a matter of sending GET request to Jaggery app endpoint. No scripting required IMO. Thanks, Krishantha. [1] https://code.google.com/p/testngroove/ On Thu, Jul 10, 2014 at 12:10 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Krishantha, If it is just a dummy class as you mentioned, then that would be fine. i.e. we neither need to maintain any *.xml configurations or Java codes, other than the pom.xml configuration. Regarding my initial suggestion, you would be able to do it using [1]. [1] http://testng.org/doc/documentation-main.html#running-testng-programmatically [2] http://stackoverflow.com/questions/13732015/dynamic-test-suites-using-testng On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this.
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi Krishantha, You should be able to use the same admin service that you use for Java webapps. On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email: ruch...@wso2.com ruch...@wso2.com, blog: ruchirawageesha.blogspot.com http://ruchirawageesha.blogspot.com, mobile: +94 77 5493444 %2B94%2077%205493444* -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email: ruch...@wso2.com ruch...@wso2.com, blog: ruchirawageesha.blogspot.com http://ruchirawageesha.blogspot.com, mobile: +94 77 5493444 %2B94%2077%205493444* --
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi All, Why can't we use an annotation like @JaggeryTest rather than a Base class in this case? It will be more sophisticated to the test writers. Regards, Chamil On Wed, Jul 9, 2014 at 11:59 AM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Krishantha, You should be able to use the same admin service that you use for Java webapps. On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email: ruch...@wso2.com ruch...@wso2.com, blog: ruchirawageesha.blogspot.com http://ruchirawageesha.blogspot.com, mobile: +94 77 5493444 %2B94%2077%205493444* -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. -- *Ruchira Wageesha**Associate Technical Lead*
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi Chamil, We can introduce TestNG custom annotation easily, but I'm in confusion how custom annotation fit in to this context. Only thing which comes to my mind is include the base class to all products using TestNG annotation transform listener. Thanks, Krishantha. On Thu, Jul 10, 2014 at 10:16 AM, Chamil Jeewantha cha...@wso2.com wrote: Hi All, Why can't we use an annotation like @JaggeryTest rather than a Base class in this case? It will be more sophisticated to the test writers. Regards, Chamil On Wed, Jul 9, 2014 at 11:59 AM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Krishantha, You should be able to use the same admin service that you use for Java webapps. On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email:
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
+1. IMO we do not need to introduce any customs annotations here. We already have common tests like OSGIServerBundleStatusTest etc. written in the same way. On Thu, Jul 10, 2014 at 10:25 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Chamil, We can introduce TestNG custom annotation easily, but I'm in confusion how custom annotation fit in to this context. Only thing which comes to my mind is include the base class to all products using TestNG annotation transform listener. Thanks, Krishantha. On Thu, Jul 10, 2014 at 10:16 AM, Chamil Jeewantha cha...@wso2.com wrote: Hi All, Why can't we use an annotation like @JaggeryTest rather than a Base class in this case? It will be more sophisticated to the test writers. Regards, Chamil On Wed, Jul 9, 2014 at 11:59 AM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Krishantha, You should be able to use the same admin service that you use for Java webapps. On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc.
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
AFAIU, If any test is extended from the base class, you identify it as a jaggery test and do some special treatment like integrating to Emma etc... The annotation marks the class as qualified to receive the special treatment you are going to do within the abstract class. The best example is the difference between JUnit 3 4. In JUnit 3 we had to extend our test class from junit.framework.TestCase class. But in Junit 4, the abstract class has been deprecated and introduced a simple annotations. So I think asking people to implement their test classes from a Abstract class is an old fashion which was followed before java 5. We can improve the readability of the test using annotations. On Thu, Jul 10, 2014 at 10:25 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Chamil, We can introduce TestNG custom annotation easily, but I'm in confusion how custom annotation fit in to this context. Only thing which comes to my mind is include the base class to all products using TestNG annotation transform listener. Thanks, Krishantha. On Thu, Jul 10, 2014 at 10:16 AM, Chamil Jeewantha cha...@wso2.com wrote: Hi All, Why can't we use an annotation like @JaggeryTest rather than a Base class in this case? It will be more sophisticated to the test writers. Regards, Chamil On Wed, Jul 9, 2014 at 11:59 AM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Krishantha, You should be able to use the same admin service that you use for Java webapps. On Wed, Jul 9, 2014 at 11:20 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi,
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi Ruchira, On Mon, Jul 7, 2014 at 6:39 PM, Ruchira Wageesha ruch...@wso2.com wrote: On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml Idea of this new feature was to run jaggery tests and display the result in TestNG report generated by test framework. To implement the plugin you mentioned, we need to write a wrapper plugin around surefire plugin. I don't know whether this capability is provided by maven. The test class that developers going write will be look like this [1]. It is matter of extending abstract test class. Will do a demo on this and discuss with you. So we can decide what is the best way to go ahead. [1] https://github.com/wso2/product-dss/blob/master/modules/integration/tests-integration/tests/src/test/java/org/wso2/dss/integration/test/server/mgt/DSSServerBundleStatusTestCase.java again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. If we refer carbon.home in tests case, then the test case will have to be skipped from platform/distributed scenarios (where distribution is hosted remotely). We refer carbon.home only if there is no way to get test inputs. Is there a admin service to get all jaggery apps deployed on carbon instance ? Thanks, Krishantha. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email: ruch...@wso2.com ruch...@wso2.com, blog: ruchirawageesha.blogspot.com http://ruchirawageesha.blogspot.com, mobile: +94 77 5493444 %2B94%2077%205493444* -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email: ruch...@wso2.com ruch...@wso2.com, blog: ruchirawageesha.blogspot.com http://ruchirawageesha.blogspot.com, mobile: +94 77 5493444 %2B94%2077%205493444* -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing
[Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email: ruch...@wso2.com ruch...@wso2.com, blog: ruchirawageesha.blogspot.com http://ruchirawageesha.blogspot.com, mobile: +94 77 5493444* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email: ruch...@wso2.com ruch...@wso2.com, blog: ruchirawageesha.blogspot.com http://ruchirawageesha.blogspot.com, mobile: +94 77 5493444 %2B94%2077%205493444* -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Plug Jaggery test framework to automation framework - Executing jaggery test cases during the build time and bridging it with test framework
On Mon, Jul 7, 2014 at 4:46 PM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Ruchira, This is not going to be a simple plugin to implement.This plugin will act as a test executor and report generator, and we need to integrate code coverage generation as well. Maven already provide sure-fire plug for test execution and TestNG and Emma provide other capabilities. So writing a new plugin from the scratch will be more work and we are going to ignore existing plugins and test framework capabilities. I think we are trying to reinvent the wheel Sorry Krishantha, you have got it wrong :). I never said to reinvent the wheel and I know it is useless. To clarify what I meant, assume the configuration that everyone put in their pom.xml asking TestNG/Emma to execute the test class that they wrote(Let's say it as X and everyone will have to write the same class). Instead of that, your test framework jar bundles the class(X) and you write a maven plugin, which allows someone to ask TestNG/Emma to execute tests with your bundled class(X) by just specifying maven plugin configuration in their pom.xml again. Adding one single test class just by extending a class provided by test framework doesn't going to be overkill. Is there a way to discover all jaggery applications hosted in carbon server instance? if jaggery app discovery is available, then we can use testNG listener to run all unit tests in those Jaggery apps even without writing single java class in product test modules. +1. If you list down all the directories available within repository/deployment/server/jaggeryapps directory, those are the Jaggery apps available. Thanks, Krishantha. On Mon, Jul 7, 2014 at 4:31 PM, Ruchira Wageesha ruch...@wso2.com wrote: Hi Dimuthu, With this approach, everyone who wants to test Jaggery apps will have to write the same Java code by extending your base class, which is redundant. Hence, IMO, the best options would be to wrap your base class and write a maven plugin. So, what other's will have to do is, just add that configuration in the pom and tests will be executed automatically without writing any Java code. /Ruchira On Mon, Jul 7, 2014 at 3:39 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi, I have started working on this. Our plan is to write a common test class inside carbon-platform-integration-utils module which product teams can extend and start running jaggery tests residing inside [PRODUCT_HOME]/repository/deployment/server/jaggeryapps/xxx/test directory. Invocation of each test endpoint will be treated as a separate test case resulting number of test cases to be executed will get equal to number of test endpoints to be invoked. Regards Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email: ruch...@wso2.com ruch...@wso2.com, blog: ruchirawageesha.blogspot.com http://ruchirawageesha.blogspot.com, mobile: +94 77 5493444 %2B94%2077%205493444* -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com http://wso2.com* *email: ruch...@wso2.com ruch...@wso2.com, blog: ruchirawageesha.blogspot.com http://ruchirawageesha.blogspot.com, mobile: +94 77 5493444* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
