Re: [Dev] [DEV] Error when Running Automation Tests in Parallel

[Akalanka Pagoda Arachchi]

Hi,

I've tried setting parallel scope to "classes". However, it also does not
seem to help. This issue did not occur when I reduced the number of
parallel threads to '2' instead of '10'. Therefore, as a resolution I kept
the number of parallel threads at '2' and changed the parallel scope to
"classes" as well.

Also I've updated the testNg version to 6.9.9.

Thanks,
Akalanka.

On Thu, Nov 5, 2015 at 5:38 PM, Malintha Adikari  wrote:

> Hi,
> Testng provides 4 modes to run your tests in parallel[1]. Among those 4
> modes, we recommend parallel="classes" mode.
> 
>
> *parallel="methods"*: TestNG will run all your test methods in separate
> threads. Dependent methods will also run in separate threads but they will
> respect the order that you specified. If you use this mode, there is a
> possibility to run your test's @BeforeClass methods in different threads
> than the @Test methods. According to Test Framework architecture, we use
> variables which are initialized inside @BeforeClass methods inside our
> @Test methods. So executing these two methods in separate threads may cause
> problems. So we do not recommend to use this mode.
>
> *parallel="classes"*: TestNG will run all the methods in the same class
> in the same thread, but each class will be run in a separate thread. In
> this mode all methods inside your test class (@BeforeClass , @Test,
> @AfterClass) will be executed in one thread. It guarantees the values which
> were assigned to variables inside @BeforeClass method are available to
> @Test methods as expected. We recommend this mode to run your test in
> parallel.
>
>
> [1] http://testng.org/doc/documentation-main.html#parallel-running
> 
>
> Regards,
> Malintha
>
>
> On Thu, Nov 5, 2015 at 2:41 PM, Akalanka Pagoda Arachchi <
> darsha...@wso2.com> wrote:
>
>> Thanks Dharshana.
>>
>> I will modify the tests as you have suggested and see if it fixes the
>> issue.
>>
>> Thanks,
>> Akalanka.
>>
>> On Thu, Nov 5, 2015 at 2:37 PM, Dharshana Warusavitharana <
>> dharsha...@wso2.com> wrote:
>>
>>> Hi Akalanka,
>>>
>>> I see some issue how you initiate parallel test run in testng.xml
>>>
>>> >> thread-count="10">
>>>
>>> parallel= true is mainly used in test method level to run test methods
>>> parallel once you add this to suite level(testNG.xml) it will trigger all
>>> test methods in test suites run parallel. I dont think our architecture can
>>> support this.
>>>
>>> So what you can do is run tests parallel in class level, you can refer
>>> tag below to support that.
>>>
>>> <*suite* name="tests-amqp-parallel" parallel="classes"
>>> thread-count="10" >
>>> And please update testNg version of your test suite above 6.8 (latest
>>> is 6.9.9) where lots of improvements were introduced in those later
>>> versions.
>>>
>>> And other thing you can try our is make test suites run parallel.
>>>
>>> 
>>>
>>> for this you can add several test tags inside testng.xml and those test
>>> suites will run in parallel.
>>>
>>> Thank you,
>>> Dharshana.
>>>
>>> On Thu, Nov 5, 2015 at 2:21 PM, Akalanka Pagoda Arachchi <
>>> darsha...@wso2.com> wrote:
>>>
 Hi Malitha,

 Thanks for taking a look at this.

 All our test cases are extended from MBIntegrationBaseTest [1]. Within
 each test's prepare method, we invoke the 'init' method of this class. The
 exception is occuring when invoking the line
 backendURL = automationContext.getContextUrls().
 getBackEndUrl();

 We have added a synchronise block there as well trying to rectify this
 issue.

 [1] -
 https://github.com/wso2/product-mb/blob/master/modules/integration/tests-common/integration-tests-utils/src/main/java/org/wso2/mb/integration/common/utils/backend/MBIntegrationBaseTest.java

 Thanks,
 Akalanka.

 On Thu, Nov 5, 2015 at 2:12 PM, Malintha Adikari 
 wrote:

> Thank you. I will have a look.
>
> On Thu, Nov 5, 2015 at 2:10 PM, Akalanka Pagoda Arachchi <
> darsha...@wso2.com> wrote:
>
>> Hi Malintha,
>>
>> The testng.xml file is located in [1].
>>
>> The test cases failing are different in each run, at the moment test
>> [2], and [3] have failed in jenkins.
>>
>> Please note that although this is frequent in jenkins, when doing a
>> local run, this issue is rarely occurring. Furthermore, this issue did 
>> not
>> occur when the tests are run sequentially. We made it to run in parallel
>> since it will save more time when building.
>>
>> [1] -
>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/resources/testng.xml
>> [2] -
>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/AutoAcknowledgementsTestCase.java
>> [3] -

Re: [Dev] [DEV] Error when Running Automation Tests in Parallel

[Akalanka Pagoda Arachchi]

Hi Malintha,

The testng.xml file is located in [1].

The test cases failing are different in each run, at the moment test [2],
and [3] have failed in jenkins.

Please note that although this is frequent in jenkins, when doing a local
run, this issue is rarely occurring. Furthermore, this issue did not occur
when the tests are run sequentially. We made it to run in parallel since it
will save more time when building.

[1] -
https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/resources/testng.xml
[2] -
https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/AutoAcknowledgementsTestCase.java
[3] -
https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/DurableTopicSubscriptionTestCase.java

Thanks,
Akalanka.


On Thu, Nov 5, 2015 at 1:51 PM, Malintha Adikari  wrote:

> Hi Akalanka,
>
> Can you point me to the test case which you are trying to run in parallel.
>
> Regards,
> Malintha
>
> On Thu, Nov 5, 2015 at 11:47 AM, Akalanka Pagoda Arachchi <
> darsha...@wso2.com> wrote:
>
>> Hi All,
>>
>> When we configure to run automation tests in parallel, following error is
>> thrown intermittently.
>>
>> I"ve enabled to run 10 threads in parallel with the following config in
>> testng.xml file.
>>
>> > thread-count="10">
>>
>> Stacktrace
>>
>> java.lang.NullPointerException
>>  at org.apache.xerces.dom.ParentNode.nodeListGetLength(Unknown Source)
>>  at org.apache.xerces.dom.ParentNode.getLength(Unknown Source)
>>  at 
>> org.wso2.carbon.automation.engine.context.AutomationContext.extractUser(AutomationContext.java:307)
>>  at 
>> org.wso2.carbon.automation.engine.context.AutomationContext.getSuperTenant(AutomationContext.java:286)
>>  at 
>> org.wso2.carbon.automation.engine.context.AutomationContext.getContextTenant(AutomationContext.java:336)
>>  at 
>> org.wso2.carbon.automation.engine.context.AutomationContext.getContextUrls(AutomationContext.java:477)
>>  at 
>> org.wso2.mb.integration.common.utils.backend.MBIntegrationBaseTest.init(MBIntegrationBaseTest.java:59)
>>  at 
>> org.wso2.mb.integration.tests.amqp.functional.AutoAcknowledgementsTestCase.prepare(AutoAcknowledgementsTestCase.java:61)
>>
>>
>> There are kind of similar issue reported regarding DOM usage with threads 
>> [1]. This is happening when initialising each test with following two lines.
>>
>>
>> automationContext = new AutomationContext("MB", userMode);
>>
>> backendURL = automationContext.getContextUrls().getBackEndUrl();
>>
>>
>> Any idea on this?
>>
>>
>> Thanks,
>>
>> Akalanka
>>
>>
>> [1] - https://issues.apache.org/jira/browse/XERCESJ-727
>>
>>
>>
>>
>> --
>> *Darshana Akalanka Pagoda Arachchi,*
>> *Software Engineer*
>> *078-4721791*
>>
>
>
>
> --
> *Malintha Adikari*
> Software Engineer
> WSO2 Inc.; http://wso2.com
> lean.enterprise.middleware
>
> Mobile: +94 71 2312958
> Blog:http://malinthas.blogspot.com
> Page:   http://about.me/malintha
>



-- 
*Darshana Akalanka Pagoda Arachchi,*
*Software Engineer*
*078-4721791*
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DEV] Error when Running Automation Tests in Parallel

[Malintha Adikari]

Hi Akalanka,

Can you point me to the test case which you are trying to run in parallel.

Regards,
Malintha

On Thu, Nov 5, 2015 at 11:47 AM, Akalanka Pagoda Arachchi <
darsha...@wso2.com> wrote:

> Hi All,
>
> When we configure to run automation tests in parallel, following error is
> thrown intermittently.
>
> I"ve enabled to run 10 threads in parallel with the following config in
> testng.xml file.
>
>  thread-count="10">
>
> Stacktrace
>
> java.lang.NullPointerException
>   at org.apache.xerces.dom.ParentNode.nodeListGetLength(Unknown Source)
>   at org.apache.xerces.dom.ParentNode.getLength(Unknown Source)
>   at 
> org.wso2.carbon.automation.engine.context.AutomationContext.extractUser(AutomationContext.java:307)
>   at 
> org.wso2.carbon.automation.engine.context.AutomationContext.getSuperTenant(AutomationContext.java:286)
>   at 
> org.wso2.carbon.automation.engine.context.AutomationContext.getContextTenant(AutomationContext.java:336)
>   at 
> org.wso2.carbon.automation.engine.context.AutomationContext.getContextUrls(AutomationContext.java:477)
>   at 
> org.wso2.mb.integration.common.utils.backend.MBIntegrationBaseTest.init(MBIntegrationBaseTest.java:59)
>   at 
> org.wso2.mb.integration.tests.amqp.functional.AutoAcknowledgementsTestCase.prepare(AutoAcknowledgementsTestCase.java:61)
>
>
> There are kind of similar issue reported regarding DOM usage with threads 
> [1]. This is happening when initialising each test with following two lines.
>
>
> automationContext = new AutomationContext("MB", userMode);
>
> backendURL = automationContext.getContextUrls().getBackEndUrl();
>
>
> Any idea on this?
>
>
> Thanks,
>
> Akalanka
>
>
> [1] - https://issues.apache.org/jira/browse/XERCESJ-727
>
>
>
>
> --
> *Darshana Akalanka Pagoda Arachchi,*
> *Software Engineer*
> *078-4721791*
>



-- 
*Malintha Adikari*
Software Engineer
WSO2 Inc.; http://wso2.com
lean.enterprise.middleware

Mobile: +94 71 2312958
Blog:http://malinthas.blogspot.com
Page:   http://about.me/malintha
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DEV] Error when Running Automation Tests in Parallel

[Malintha Adikari]

Thank you. I will have a look.

On Thu, Nov 5, 2015 at 2:10 PM, Akalanka Pagoda Arachchi  wrote:

> Hi Malintha,
>
> The testng.xml file is located in [1].
>
> The test cases failing are different in each run, at the moment test [2],
> and [3] have failed in jenkins.
>
> Please note that although this is frequent in jenkins, when doing a local
> run, this issue is rarely occurring. Furthermore, this issue did not occur
> when the tests are run sequentially. We made it to run in parallel since it
> will save more time when building.
>
> [1] -
> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/resources/testng.xml
> [2] -
> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/AutoAcknowledgementsTestCase.java
> [3] -
> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/DurableTopicSubscriptionTestCase.java
>
> Thanks,
> Akalanka.
>
>
> On Thu, Nov 5, 2015 at 1:51 PM, Malintha Adikari 
> wrote:
>
>> Hi Akalanka,
>>
>> Can you point me to the test case which you are trying to run in
>> parallel.
>>
>> Regards,
>> Malintha
>>
>> On Thu, Nov 5, 2015 at 11:47 AM, Akalanka Pagoda Arachchi <
>> darsha...@wso2.com> wrote:
>>
>>> Hi All,
>>>
>>> When we configure to run automation tests in parallel, following error
>>> is thrown intermittently.
>>>
>>> I"ve enabled to run 10 threads in parallel with the following config in
>>> testng.xml file.
>>>
>>> >> thread-count="10">
>>>
>>> Stacktrace
>>>
>>> java.lang.NullPointerException
>>> at org.apache.xerces.dom.ParentNode.nodeListGetLength(Unknown Source)
>>> at org.apache.xerces.dom.ParentNode.getLength(Unknown Source)
>>> at 
>>> org.wso2.carbon.automation.engine.context.AutomationContext.extractUser(AutomationContext.java:307)
>>> at 
>>> org.wso2.carbon.automation.engine.context.AutomationContext.getSuperTenant(AutomationContext.java:286)
>>> at 
>>> org.wso2.carbon.automation.engine.context.AutomationContext.getContextTenant(AutomationContext.java:336)
>>> at 
>>> org.wso2.carbon.automation.engine.context.AutomationContext.getContextUrls(AutomationContext.java:477)
>>> at 
>>> org.wso2.mb.integration.common.utils.backend.MBIntegrationBaseTest.init(MBIntegrationBaseTest.java:59)
>>> at 
>>> org.wso2.mb.integration.tests.amqp.functional.AutoAcknowledgementsTestCase.prepare(AutoAcknowledgementsTestCase.java:61)
>>>
>>>
>>> There are kind of similar issue reported regarding DOM usage with threads 
>>> [1]. This is happening when initialising each test with following two lines.
>>>
>>>
>>> automationContext = new AutomationContext("MB", userMode);
>>>
>>> backendURL = automationContext.getContextUrls().getBackEndUrl();
>>>
>>>
>>> Any idea on this?
>>>
>>>
>>> Thanks,
>>>
>>> Akalanka
>>>
>>>
>>> [1] - https://issues.apache.org/jira/browse/XERCESJ-727
>>>
>>>
>>>
>>>
>>> --
>>> *Darshana Akalanka Pagoda Arachchi,*
>>> *Software Engineer*
>>> *078-4721791*
>>>
>>
>>
>>
>> --
>> *Malintha Adikari*
>> Software Engineer
>> WSO2 Inc.; http://wso2.com
>> lean.enterprise.middleware
>>
>> Mobile: +94 71 2312958
>> Blog:http://malinthas.blogspot.com
>> Page:   http://about.me/malintha
>>
>
>
>
> --
> *Darshana Akalanka Pagoda Arachchi,*
> *Software Engineer*
> *078-4721791*
>



-- 
*Malintha Adikari*
Software Engineer
WSO2 Inc.; http://wso2.com
lean.enterprise.middleware

Mobile: +94 71 2312958
Blog:http://malinthas.blogspot.com
Page:   http://about.me/malintha
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DEV] Error when Running Automation Tests in Parallel

[Akalanka Pagoda Arachchi]

Hi Malitha,

Thanks for taking a look at this.

All our test cases are extended from MBIntegrationBaseTest [1]. Within each
test's prepare method, we invoke the 'init' method of this class. The
exception is occuring when invoking the line
backendURL = automationContext.getContextUrls().getBackEndUrl();

We have added a synchronise block there as well trying to rectify this
issue.

[1] -
https://github.com/wso2/product-mb/blob/master/modules/integration/tests-common/integration-tests-utils/src/main/java/org/wso2/mb/integration/common/utils/backend/MBIntegrationBaseTest.java

Thanks,
Akalanka.

On Thu, Nov 5, 2015 at 2:12 PM, Malintha Adikari  wrote:

> Thank you. I will have a look.
>
> On Thu, Nov 5, 2015 at 2:10 PM, Akalanka Pagoda Arachchi <
> darsha...@wso2.com> wrote:
>
>> Hi Malintha,
>>
>> The testng.xml file is located in [1].
>>
>> The test cases failing are different in each run, at the moment test [2],
>> and [3] have failed in jenkins.
>>
>> Please note that although this is frequent in jenkins, when doing a local
>> run, this issue is rarely occurring. Furthermore, this issue did not occur
>> when the tests are run sequentially. We made it to run in parallel since it
>> will save more time when building.
>>
>> [1] -
>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/resources/testng.xml
>> [2] -
>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/AutoAcknowledgementsTestCase.java
>> [3] -
>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/DurableTopicSubscriptionTestCase.java
>>
>> Thanks,
>> Akalanka.
>>
>>
>> On Thu, Nov 5, 2015 at 1:51 PM, Malintha Adikari 
>> wrote:
>>
>>> Hi Akalanka,
>>>
>>> Can you point me to the test case which you are trying to run in
>>> parallel.
>>>
>>> Regards,
>>> Malintha
>>>
>>> On Thu, Nov 5, 2015 at 11:47 AM, Akalanka Pagoda Arachchi <
>>> darsha...@wso2.com> wrote:
>>>
 Hi All,

 When we configure to run automation tests in parallel, following error
 is thrown intermittently.

 I"ve enabled to run 10 threads in parallel with the following config in
 testng.xml file.

 >>> thread-count="10">

 Stacktrace

 java.lang.NullPointerException
at org.apache.xerces.dom.ParentNode.nodeListGetLength(Unknown Source)
at org.apache.xerces.dom.ParentNode.getLength(Unknown Source)
at 
 org.wso2.carbon.automation.engine.context.AutomationContext.extractUser(AutomationContext.java:307)
at 
 org.wso2.carbon.automation.engine.context.AutomationContext.getSuperTenant(AutomationContext.java:286)
at 
 org.wso2.carbon.automation.engine.context.AutomationContext.getContextTenant(AutomationContext.java:336)
at 
 org.wso2.carbon.automation.engine.context.AutomationContext.getContextUrls(AutomationContext.java:477)
at 
 org.wso2.mb.integration.common.utils.backend.MBIntegrationBaseTest.init(MBIntegrationBaseTest.java:59)
at 
 org.wso2.mb.integration.tests.amqp.functional.AutoAcknowledgementsTestCase.prepare(AutoAcknowledgementsTestCase.java:61)


 There are kind of similar issue reported regarding DOM usage with threads 
 [1]. This is happening when initialising each test with following two 
 lines.


 automationContext = new AutomationContext("MB", userMode);

 backendURL = 
 automationContext.getContextUrls().getBackEndUrl();


 Any idea on this?


 Thanks,

 Akalanka


 [1] - https://issues.apache.org/jira/browse/XERCESJ-727




 --
 *Darshana Akalanka Pagoda Arachchi,*
 *Software Engineer*
 *078-4721791*

>>>
>>>
>>>
>>> --
>>> *Malintha Adikari*
>>> Software Engineer
>>> WSO2 Inc.; http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> Mobile: +94 71 2312958
>>> Blog:http://malinthas.blogspot.com
>>> Page:   http://about.me/malintha
>>>
>>
>>
>>
>> --
>> *Darshana Akalanka Pagoda Arachchi,*
>> *Software Engineer*
>> *078-4721791*
>>
>
>
>
> --
> *Malintha Adikari*
> Software Engineer
> WSO2 Inc.; http://wso2.com
> lean.enterprise.middleware
>
> Mobile: +94 71 2312958
> Blog:http://malinthas.blogspot.com
> Page:   http://about.me/malintha
>



-- 
*Darshana Akalanka Pagoda Arachchi,*
*Software Engineer*
*078-4721791*
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DEV] Error when Running Automation Tests in Parallel

[Dharshana Warusavitharana]

Hi Akalanka,

I see some issue how you initiate parallel test run in testng.xml



parallel= true is mainly used in test method level to run test methods
parallel once you add this to suite level(testNG.xml) it will trigger all
test methods in test suites run parallel. I dont think our architecture can
support this.

So what you can do is run tests parallel in class level, you can refer tag
below to support that.

<*suite* name="tests-amqp-parallel" parallel="classes" thread-count="10" >
And please update testNg version of your test suite above 6.8 (latest
is 6.9.9) where lots of improvements were introduced in those later
versions.

And other thing you can try our is make test suites run parallel.



for this you can add several test tags inside testng.xml and those test
suites will run in parallel.

Thank you,
Dharshana.

On Thu, Nov 5, 2015 at 2:21 PM, Akalanka Pagoda Arachchi  wrote:

> Hi Malitha,
>
> Thanks for taking a look at this.
>
> All our test cases are extended from MBIntegrationBaseTest [1]. Within
> each test's prepare method, we invoke the 'init' method of this class. The
> exception is occuring when invoking the line
> backendURL = automationContext.getContextUrls().getBackEndUrl();
>
> We have added a synchronise block there as well trying to rectify this
> issue.
>
> [1] -
> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-common/integration-tests-utils/src/main/java/org/wso2/mb/integration/common/utils/backend/MBIntegrationBaseTest.java
>
> Thanks,
> Akalanka.
>
> On Thu, Nov 5, 2015 at 2:12 PM, Malintha Adikari 
> wrote:
>
>> Thank you. I will have a look.
>>
>> On Thu, Nov 5, 2015 at 2:10 PM, Akalanka Pagoda Arachchi <
>> darsha...@wso2.com> wrote:
>>
>>> Hi Malintha,
>>>
>>> The testng.xml file is located in [1].
>>>
>>> The test cases failing are different in each run, at the moment test
>>> [2], and [3] have failed in jenkins.
>>>
>>> Please note that although this is frequent in jenkins, when doing a
>>> local run, this issue is rarely occurring. Furthermore, this issue did not
>>> occur when the tests are run sequentially. We made it to run in parallel
>>> since it will save more time when building.
>>>
>>> [1] -
>>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/resources/testng.xml
>>> [2] -
>>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/AutoAcknowledgementsTestCase.java
>>> [3] -
>>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/DurableTopicSubscriptionTestCase.java
>>>
>>> Thanks,
>>> Akalanka.
>>>
>>>
>>> On Thu, Nov 5, 2015 at 1:51 PM, Malintha Adikari 
>>> wrote:
>>>
 Hi Akalanka,

 Can you point me to the test case which you are trying to run in
 parallel.

 Regards,
 Malintha

 On Thu, Nov 5, 2015 at 11:47 AM, Akalanka Pagoda Arachchi <
 darsha...@wso2.com> wrote:

> Hi All,
>
> When we configure to run automation tests in parallel, following error
> is thrown intermittently.
>
> I"ve enabled to run 10 threads in parallel with the following config
> in testng.xml file.
>
>  parallel="true" thread-count="10">
>
> Stacktrace
>
> java.lang.NullPointerException
>   at org.apache.xerces.dom.ParentNode.nodeListGetLength(Unknown Source)
>   at org.apache.xerces.dom.ParentNode.getLength(Unknown Source)
>   at 
> org.wso2.carbon.automation.engine.context.AutomationContext.extractUser(AutomationContext.java:307)
>   at 
> org.wso2.carbon.automation.engine.context.AutomationContext.getSuperTenant(AutomationContext.java:286)
>   at 
> org.wso2.carbon.automation.engine.context.AutomationContext.getContextTenant(AutomationContext.java:336)
>   at 
> org.wso2.carbon.automation.engine.context.AutomationContext.getContextUrls(AutomationContext.java:477)
>   at 
> org.wso2.mb.integration.common.utils.backend.MBIntegrationBaseTest.init(MBIntegrationBaseTest.java:59)
>   at 
> org.wso2.mb.integration.tests.amqp.functional.AutoAcknowledgementsTestCase.prepare(AutoAcknowledgementsTestCase.java:61)
>
>
> There are kind of similar issue reported regarding DOM usage with threads 
> [1]. This is happening when initialising each test with following two 
> lines.
>
>
> automationContext = new AutomationContext("MB", userMode);
>
> backendURL = 
> automationContext.getContextUrls().getBackEndUrl();
>
>
> Any idea on this?
>
>
> Thanks,
>
> Akalanka
>
>
> [1] - https://issues.apache.org/jira/browse/XERCESJ-727
>
>
>
>
> --
> *Darshana Akalanka 

Re: [Dev] [DEV] Error when Running Automation Tests in Parallel

[Akalanka Pagoda Arachchi]

Thanks Dharshana.

I will modify the tests as you have suggested and see if it fixes the issue.

Thanks,
Akalanka.

On Thu, Nov 5, 2015 at 2:37 PM, Dharshana Warusavitharana <
dharsha...@wso2.com> wrote:

> Hi Akalanka,
>
> I see some issue how you initiate parallel test run in testng.xml
>
>  thread-count="10">
>
> parallel= true is mainly used in test method level to run test methods
> parallel once you add this to suite level(testNG.xml) it will trigger all
> test methods in test suites run parallel. I dont think our architecture can
> support this.
>
> So what you can do is run tests parallel in class level, you can refer tag
> below to support that.
>
> <*suite* name="tests-amqp-parallel" parallel="classes" thread-count="10" >
> And please update testNg version of your test suite above 6.8 (latest
> is 6.9.9) where lots of improvements were introduced in those later
> versions.
>
> And other thing you can try our is make test suites run parallel.
>
> 
>
> for this you can add several test tags inside testng.xml and those test
> suites will run in parallel.
>
> Thank you,
> Dharshana.
>
> On Thu, Nov 5, 2015 at 2:21 PM, Akalanka Pagoda Arachchi <
> darsha...@wso2.com> wrote:
>
>> Hi Malitha,
>>
>> Thanks for taking a look at this.
>>
>> All our test cases are extended from MBIntegrationBaseTest [1]. Within
>> each test's prepare method, we invoke the 'init' method of this class. The
>> exception is occuring when invoking the line
>> backendURL = automationContext.getContextUrls().getBackEndUrl();
>>
>> We have added a synchronise block there as well trying to rectify this
>> issue.
>>
>> [1] -
>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-common/integration-tests-utils/src/main/java/org/wso2/mb/integration/common/utils/backend/MBIntegrationBaseTest.java
>>
>> Thanks,
>> Akalanka.
>>
>> On Thu, Nov 5, 2015 at 2:12 PM, Malintha Adikari 
>> wrote:
>>
>>> Thank you. I will have a look.
>>>
>>> On Thu, Nov 5, 2015 at 2:10 PM, Akalanka Pagoda Arachchi <
>>> darsha...@wso2.com> wrote:
>>>
 Hi Malintha,

 The testng.xml file is located in [1].

 The test cases failing are different in each run, at the moment test
 [2], and [3] have failed in jenkins.

 Please note that although this is frequent in jenkins, when doing a
 local run, this issue is rarely occurring. Furthermore, this issue did not
 occur when the tests are run sequentially. We made it to run in parallel
 since it will save more time when building.

 [1] -
 https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/resources/testng.xml
 [2] -
 https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/AutoAcknowledgementsTestCase.java
 [3] -
 https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/DurableTopicSubscriptionTestCase.java

 Thanks,
 Akalanka.


 On Thu, Nov 5, 2015 at 1:51 PM, Malintha Adikari 
 wrote:

> Hi Akalanka,
>
> Can you point me to the test case which you are trying to run in
> parallel.
>
> Regards,
> Malintha
>
> On Thu, Nov 5, 2015 at 11:47 AM, Akalanka Pagoda Arachchi <
> darsha...@wso2.com> wrote:
>
>> Hi All,
>>
>> When we configure to run automation tests in parallel, following
>> error is thrown intermittently.
>>
>> I"ve enabled to run 10 threads in parallel with the following config
>> in testng.xml file.
>>
>> > parallel="true" thread-count="10">
>>
>> Stacktrace
>>
>> java.lang.NullPointerException
>>  at org.apache.xerces.dom.ParentNode.nodeListGetLength(Unknown Source)
>>  at org.apache.xerces.dom.ParentNode.getLength(Unknown Source)
>>  at 
>> org.wso2.carbon.automation.engine.context.AutomationContext.extractUser(AutomationContext.java:307)
>>  at 
>> org.wso2.carbon.automation.engine.context.AutomationContext.getSuperTenant(AutomationContext.java:286)
>>  at 
>> org.wso2.carbon.automation.engine.context.AutomationContext.getContextTenant(AutomationContext.java:336)
>>  at 
>> org.wso2.carbon.automation.engine.context.AutomationContext.getContextUrls(AutomationContext.java:477)
>>  at 
>> org.wso2.mb.integration.common.utils.backend.MBIntegrationBaseTest.init(MBIntegrationBaseTest.java:59)
>>  at 
>> org.wso2.mb.integration.tests.amqp.functional.AutoAcknowledgementsTestCase.prepare(AutoAcknowledgementsTestCase.java:61)
>>
>>
>> There are kind of similar issue reported regarding DOM usage with 
>> threads [1]. This is happening when initialising each test with 
>> following two lines.
>>
>>
>> 

Re: [Dev] [DEV] Error when Running Automation Tests in Parallel

[Malintha Adikari]

Hi,
Testng provides 4 modes to run your tests in parallel[1]. Among those 4
modes, we recommend parallel="classes" mode.


*parallel="methods"*: TestNG will run all your test methods in separate
threads. Dependent methods will also run in separate threads but they will
respect the order that you specified. If you use this mode, there is a
possibility to run your test's @BeforeClass methods in different threads
than the @Test methods. According to Test Framework architecture, we use
variables which are initialized inside @BeforeClass methods inside our
@Test methods. So executing these two methods in separate threads may cause
problems. So we do not recommend to use this mode.

*parallel="classes"*: TestNG will run all the methods in the same class in
the same thread, but each class will be run in a separate thread. In this
mode all methods inside your test class (@BeforeClass , @Test, @AfterClass)
will be executed in one thread. It guarantees the values which were
assigned to variables inside @BeforeClass method are available to @Test
methods as expected. We recommend this mode to run your test in parallel.


[1] http://testng.org/doc/documentation-main.html#parallel-running


Regards,
Malintha


On Thu, Nov 5, 2015 at 2:41 PM, Akalanka Pagoda Arachchi  wrote:

> Thanks Dharshana.
>
> I will modify the tests as you have suggested and see if it fixes the
> issue.
>
> Thanks,
> Akalanka.
>
> On Thu, Nov 5, 2015 at 2:37 PM, Dharshana Warusavitharana <
> dharsha...@wso2.com> wrote:
>
>> Hi Akalanka,
>>
>> I see some issue how you initiate parallel test run in testng.xml
>>
>> > thread-count="10">
>>
>> parallel= true is mainly used in test method level to run test methods
>> parallel once you add this to suite level(testNG.xml) it will trigger all
>> test methods in test suites run parallel. I dont think our architecture can
>> support this.
>>
>> So what you can do is run tests parallel in class level, you can refer
>> tag below to support that.
>>
>> <*suite* name="tests-amqp-parallel" parallel="classes" thread-count="10"
>> >
>> And please update testNg version of your test suite above 6.8 (latest
>> is 6.9.9) where lots of improvements were introduced in those later
>> versions.
>>
>> And other thing you can try our is make test suites run parallel.
>>
>> 
>>
>> for this you can add several test tags inside testng.xml and those test
>> suites will run in parallel.
>>
>> Thank you,
>> Dharshana.
>>
>> On Thu, Nov 5, 2015 at 2:21 PM, Akalanka Pagoda Arachchi <
>> darsha...@wso2.com> wrote:
>>
>>> Hi Malitha,
>>>
>>> Thanks for taking a look at this.
>>>
>>> All our test cases are extended from MBIntegrationBaseTest [1]. Within
>>> each test's prepare method, we invoke the 'init' method of this class. The
>>> exception is occuring when invoking the line
>>> backendURL = automationContext.getContextUrls().getBackEndUrl();
>>>
>>> We have added a synchronise block there as well trying to rectify this
>>> issue.
>>>
>>> [1] -
>>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-common/integration-tests-utils/src/main/java/org/wso2/mb/integration/common/utils/backend/MBIntegrationBaseTest.java
>>>
>>> Thanks,
>>> Akalanka.
>>>
>>> On Thu, Nov 5, 2015 at 2:12 PM, Malintha Adikari 
>>> wrote:
>>>
 Thank you. I will have a look.

 On Thu, Nov 5, 2015 at 2:10 PM, Akalanka Pagoda Arachchi <
 darsha...@wso2.com> wrote:

> Hi Malintha,
>
> The testng.xml file is located in [1].
>
> The test cases failing are different in each run, at the moment test
> [2], and [3] have failed in jenkins.
>
> Please note that although this is frequent in jenkins, when doing a
> local run, this issue is rarely occurring. Furthermore, this issue did not
> occur when the tests are run sequentially. We made it to run in parallel
> since it will save more time when building.
>
> [1] -
> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/resources/testng.xml
> [2] -
> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/AutoAcknowledgementsTestCase.java
> [3] -
> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/DurableTopicSubscriptionTestCase.java
>
> Thanks,
> Akalanka.
>
>
> On Thu, Nov 5, 2015 at 1:51 PM, Malintha Adikari 
> wrote:
>
>> Hi Akalanka,
>>
>> Can you point me to the test case which you are trying to run in
>> parallel.
>>
>> Regards,
>> Malintha
>>
>> On Thu, Nov 5, 2015 at 11:47 AM, Akalanka Pagoda Arachchi <
>> darsha...@wso2.com> wrote:
>>
>>> 

[Dev] [DEV] Error when Running Automation Tests in Parallel

[Akalanka Pagoda Arachchi]

Hi All,

When we configure to run automation tests in parallel, following error is
thrown intermittently.

I"ve enabled to run 10 threads in parallel with the following config in
testng.xml file.



Stacktrace

java.lang.NullPointerException
at org.apache.xerces.dom.ParentNode.nodeListGetLength(Unknown Source)
at org.apache.xerces.dom.ParentNode.getLength(Unknown Source)
at 
org.wso2.carbon.automation.engine.context.AutomationContext.extractUser(AutomationContext.java:307)
at 
org.wso2.carbon.automation.engine.context.AutomationContext.getSuperTenant(AutomationContext.java:286)
at 
org.wso2.carbon.automation.engine.context.AutomationContext.getContextTenant(AutomationContext.java:336)
at 
org.wso2.carbon.automation.engine.context.AutomationContext.getContextUrls(AutomationContext.java:477)
at 
org.wso2.mb.integration.common.utils.backend.MBIntegrationBaseTest.init(MBIntegrationBaseTest.java:59)
at 
org.wso2.mb.integration.tests.amqp.functional.AutoAcknowledgementsTestCase.prepare(AutoAcknowledgementsTestCase.java:61)


There are kind of similar issue reported regarding DOM usage with
threads [1]. This is happening when initialising each test with
following two lines.


automationContext = new AutomationContext("MB", userMode);

backendURL = automationContext.getContextUrls().getBackEndUrl();


Any idea on this?


Thanks,

Akalanka


[1] - https://issues.apache.org/jira/browse/XERCESJ-727




-- 
*Darshana Akalanka Pagoda Arachchi,*
*Software Engineer*
*078-4721791*
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev