We had a debug session to find the root cause. The issue with incorrect usage with depends on annotation in SchemaAddTestCase. Only one test case depends on another case and rest of the test case execution happened in random order.
After putting depends on annotation and getting the tests to run in order solve the problem. We might need to document this behavior with TestNG in our wiki. I will check whether this is TestNG bug or not. Thanks, Krishantha. On Fri, Jul 11, 2014 at 2:18 PM, Sameera Kannangara <samee...@wso2.com> wrote: > HI Krishantha, > > Please find TA JIRA created at [1] <https://wso2.org/jira/browse/TA-857>. > We will exclude test failing due to interleaving, when taking test > summaries. > > Thank you, > SameeraK > > [1] https://wso2.org/jira/browse/TA-857 > > > > > On Fri, Jul 11, 2014 at 12:57 PM, Krishantha Samaraweera < > krishan...@wso2.com> wrote: > >> Hi Sameera, >> >> It will take bit of time to detect root cause of this problem. So can you >> disable the interleaving test cases and create TA JIRA to track it. >> >> Thanks, >> Krishantha. >> >> >> On Fri, Jul 11, 2014 at 12:12 AM, Krishantha Samaraweera < >> krishan...@wso2.com> wrote: >> >>> Hi all, >>> >>> You can find reasons for this kind of intermittent issue at TAF wiki - >>> http://docs.wso2.com/display/TA430/FAQ#FAQ-Whydoesmytestmodulecontainintermittenttestfailures? >>> >>> I suspect this issue is something to do with test resource cleanup. Need >>> to check run all tests and do extensive debug session. >>> >>> Please share active GIT repo URL. >>> >>> Thanks, >>> Krishantha. >>> >>> >>> On Thu, Jul 10, 2014 at 4:46 PM, Subash Chaturanga <sub...@wso2.com> >>> wrote: >>> >>>> Adding Krishantha >>>> >>>> >>>> On Thu, Jul 10, 2014 at 3:57 PM, Sameera Kannangara <samee...@wso2.com> >>>> wrote: >>>> >>>>> Hi all, >>>>> >>>>> When running G-Reg tests I've encountered several test cases which >>>>> passes when the containing class executed individually and fails when >>>>> tests >>>>> are run as packages. Following are few examples. >>>>> >>>>> >>>>> deleteResources(org.wso2.carbon.registry.metadata.test.schema.SchemaAddTestCase) >>>>> >>>>> addAssociationToRoot(org.wso2.carbon.registry.ws.client.test.general.old.TestAssociationWSTestCase) >>>>> >>>>> cleanup(org.wso2.carbon.registry.ws.client.test.general.old.TestAssociationWSTestCase) >>>>> >>>>> When remote debugging these test classes it is observed that tests >>>>> from different classes run interleaved, and that some times lead to >>>>> changing and removing shared resources, results in test failures. >>>>> >>>>> How can we solve this problem? >>>>> >>>>> Thank you, >>>>> SameeraK >>>>> >>>>> >>>>> -- >>>>> Sameera Kannangara, >>>>> Software Engineer, >>>>> WSO2 Inc. >>>>> >>>>> _______________________________________________ >>>>> Dev mailing list >>>>> Dev@wso2.org >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> Thanks >>>> /subash >>>> >>>> *Subash Chaturanga* >>>> Senior Software Engineer & Lead WSO2 Governance Registry >>>> Platform TG; WSO2 Inc. http://wso2.com >>>> Contact: >>>> email: sub...@wso2.com >>>> blog: http://subashsdm.blogspot.com/ >>>> twitter: @subash89 >>>> phone: +9477 2225922 >>>> Lean . Enterprise . Middleware >>>> >>> >>> >>> >>> -- >>> Krishantha Samaraweera >>> Senior Technical Lead - Test Automation >>> Mobile: +94 77 7759918 >>> WSO2, Inc.; http://wso2.com/ >>> lean . enterprise . middlewear. >>> >> >> >> >> -- >> Krishantha Samaraweera >> Senior Technical Lead - Test Automation >> Mobile: +94 77 7759918 >> WSO2, Inc.; http://wso2.com/ >> lean . enterprise . middlewear. >> > > > > -- > Sameera Kannangara, > Software Engineer, > WSO2 Inc. > -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear.
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
