On Feb 19, 2007, at 5:53 AM, tatyana doubtsova wrote:
The comment in auth build.xml
<!-- excluded to avoid random failures -->
is dated 07.09.06
There were several clean-up commits to
org/apache/harmony/auth/tests/module/Krb5LoginModuleTest.java
after this.
All the tests pretend to be returned were run iteratively - PASSED
All the modules with returned tests were run iteratively for 20
times in
once mode - PASSED.
In case of possible failures in iterative classlib tests runs under
cc, I'll
put the test in corresponding x-list.
I also agree that JIRA number or/and comments in x-lists would be very
helpful.
+1
On thing also to keep in mind is that tests could fail due to side
effects or VM issues that aren't clear at the time of exclusion. It
would be nice if there was a mechanical way to do this - for example,
maybe we can add a flag that also runs the excluded tests so we can
then examine the log and see what did pass that we didn't expect to.
This still feels like we're re-inventing the wheel. TestNG
supposedly has a lot of these features - has anyone tried looking at
TestNG recently? We talked about switching a while ago..
geir
Thanks,
Tanya
On 2/19/07, Alexei Zakharov <[EMAIL PROTECTED]> wrote:
At the same time there are tests that reside in exclude lists by
absolutely unknown reason. I.e. initially they were added because of
some issue that was fixed a quarter ago. So IMHO the only way to get
rid of such unwanted excludes is to run *all* tests periodically and
check do they fail anymore. This is an exhausting work - thanks
Tatyana for doing this. BTW I think it will be also nice if we put
JIRA number or any other useful comment directly to exclude lists
just
as a reminder why this test is here.
Thanks,
2007/2/19, Mikhail Loenko <[EMAIL PROTECTED]>:
> It would be good to find what were the reasons for excluding
those 12
tests.
>
> For example, the first test in the patch:
> org/apache/harmony/auth/tests/module/Krb5LoginModuleTest.java
>
> was originally excluded in build.xml with the following comment:
> <!-- The test is under development -->
> <!-- excluded to avoid random failures -->
>
> Since that the code of the test was not modified, so the test is
still
> incomplete and should not be returned back.
>
> Other tests may have similar problems too
>
> Thanks,
> Mikhail
>
> 2007/2/19, tatyana doubtsova <[EMAIL PROTECTED]>:
> > Hi,
> > I've run excluded classlib tests on win ia32 drlvm|ibmvme,
linux ia32
> > drlvm|ibmvme, linux em64t drlvm and found out that 12 tests from
common
> > x-lists passed on all 5 configurations. The tests are from
auth, luni,
> > nio_char, rmi, sound, text, lang-manegement, and security
modules.
I've
> > prepared the patch for x-lists clean-up:
> > https://issues.apache.org/jira/browse/HARMONY-3194.
Commiters, would
you,
> > please, have a look and apply, if there are no objections?
> >
> > Thanks,
> > Tanya
--
Alexei Zakharov,
Intel ESSD
