I believe there are a few more where testing individual test-suites and/or test-cases are dependent on data loaded in other test-suites and/or other test-cases.
While I don't hear/read about failing testIntegration (except where code in the base is faulty, not when test-suites/cases are faulty), I see following failures in test executions in OFBiz against jdk11: 1. Execution failed for task ':ofbiz --test component=webapp --test suitename=webapptests'. 2. Execution failed for task ':ofbiz --test component=accounting --test suitename=invoicetest'. 3. Execution failed for task ':ofbiz --test component=order --test suitename=ordertests'. 4. Execution failed for task ':ofbiz --test component=product --test suitename=producttests'. Do we have these test failing also when doing the test execution against jdk8? *Caveat: I recently set this up, so there may still be some configuration issues in the jdk11-test setup.. * Best regards, Pierre Smits *Apache Trafodion <https://trafodion.apache.org>, Vice President* *Apache Directory <https://directory.apache.org>, PMC Member* Apache Incubator <https://incubator.apache.org>, committer *Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges) since 2008* Apache Steve <https://steve.apache.org>, committer On Sat, Apr 20, 2019 at 11:40 AM Pierre Smits <pierresm...@apache.org> wrote: > Removing the code element that is causing the CI to fail is not the right > solution. It will mask the issue (and is like saying 'there is nothing > wrong with the code'), only to reappear in next iteration (when similar is > introduced). > > Loading of test data should be happening on suite level, before any > test-case is executed. > > Currently the code block in [1] is like: > > <test-suite suite-name="suite-name" ..> > > <test-case case-name="load-test data"> > > <entity-xml action="load" > entity-xml-url="component://<component-name/>testdef/data/<name>TestData.xml"/> > > </test-case> > > <test-case case-name="test-to-execute"> > > <simple-method-test > location="component://<componen-name>/<path-name>/<test-name>Test.xml"/> > > </test-case> > > </test-suite> > > > The example above treats the loading of the test data as a test-case. > Which is incorrect. Loading test data for a component's (suite of) test(s) > is not about testing the load process, but about prepping the environment > for one or more executions of test-cases of the component. > > Taking the load process to the suite level would solve this. Like in > example below: > > <test-suite suite-name="suite-name" ..> > > <!-- prep for test-cases --> > > <entity-xml action="load" > entity-xml-url="component://<component-name/>testdef/data/<name>TestData.xml"/> > > <!-- test cases --> > > <test-case case-name="test-to-execute"> > > <simple-method-test > location="component://<componen-name>/<path-name>/<test-name>Test.xml"/> > > </test-case> > > </test-suite> > > > > Best regards, > > Pierre Smits > > *Apache Trafodion <https://trafodion.apache.org>, Vice President* > *Apache Directory <https://directory.apache.org>, PMC Member* > Apache Incubator <https://incubator.apache.org>, committer > *Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges) > since 2008* > Apache Steve <https://steve.apache.org>, committer > > > On Sat, Apr 20, 2019 at 2:20 AM Mathieu Lirzin <mathieu.lir...@nereide.fr> > wrote: > >> Hello Suraj, >> >> Suraj Khurana <suraj.khur...@hotwax.co> writes: >> >> > Interestingly, I checked and got fail in a case when I have plugins as >> well >> > in the directory structure. >> > >> > I am using this command to check single test: >> > ./gradlew "ofbiz --test component=order --test suitename=ordertests >> --test >> > case=order-tests" >> >> It will not work because the “order-tests” test case implicitly depends >> on the “order-tests-data-load” (yes it sucks...). So you need to run >> the whole test suite: >> >> ./gradlew "ofbiz --test component=order --test suitename=ordertests" >> >> > It works fine when I run ./gradlew testIntegration. On a cursory view, >> IMO, >> > data is also not dependent anywhere in the plugins directory, so this >> > should not be the case. >> > >> > I will have a deeper look into it and provide more details soon >> (probably >> > during the weekend). >> >> I have been looking at this issue this evening, and after some hours I >> still do not understand the bug. It is depressing to see how hard it is >> to work with OFBiz integration tests. >> >> If you don't find a solution before the end of the week, please revert >> revision 1856596 which introduces the issue (meaning removing >> “OrderTests.groovy”). >> >> Thanks. >> >> -- >> Mathieu Lirzin >> GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37 >> >
