Re: [testing][classlib] x-lists clean-up

[Geir Magnusson Jr.]

On Feb 19, 2007, at 12:42 PM, Alexei Zakharov wrote:
Geir Magnusson Jr. wrote:
TestNG supposedly has a lot of these features - has anyone tried  
looking at
TestNG recently?  We talked about switching a while ago...

The last thing I remember was the fact that *you* can't fully enjoy
TestNG because it stores meta information inside the test source
rather than in some external resource. :) However, I agree that it
will be nice if we could get the ball moving forward here.

Yes, that's true - I can go look at one file now to see what tests  
are excluded, rather than looking at all tests to see which are :)


Alexey Petrenko wrote:
Is it possible to add comments to our exclude lists?

Sure, you can put any stuff you want into it. If it coincides with the
name of some test then such test will be excluded. Otherwise the whole
line is ignored AFAIU.

Regards,

2007/2/19, Alexey Petrenko <[EMAIL PROTECTED]>:
2007/2/19, tatyana doubtsova <[EMAIL PROTECTED]>:
> The comment in auth build.xml
> <!-- excluded to avoid random failures -->
> is dated 07.09.06
> There were several clean-up commits to
> org/apache/harmony/auth/tests/module/Krb5LoginModuleTest.java   
after this.
> All the tests pretend to be returned were run iteratively - PASSED
> All the modules with returned tests were run iteratively for 20  
times in
> once mode - PASSED.
>
> In case of possible failures in iterative classlib tests runs  
under cc, I'll
> put the test in corresponding x-list.
>
> I also agree that JIRA number or/and comments in x-lists would  
be very
> helpful.
Is it possible to add comments to our exclude lists?

SY, Alexey

> On 2/19/07, Alexei Zakharov <[EMAIL PROTECTED]> wrote:
> >
> > At the same time there are tests that reside in exclude lists by
> > absolutely unknown reason. I.e. initially they were added  
because of
> > some issue that was fixed a quarter ago. So IMHO the only way  
to get
> > rid of such unwanted excludes is to run *all* tests  
periodically and
> > check do they fail anymore.  This is an exhausting work - thanks
> > Tatyana for doing this. BTW I think it will be also nice if we  
put
> > JIRA number or any other useful comment directly to exclude  
lists just
> > as a reminder why this test is here.
> >
> > Thanks,
> >
> > 2007/2/19, Mikhail Loenko <[EMAIL PROTECTED]>:
> > > It would be good to find what were the reasons for excluding  
those 12
> > tests.
> > >
> > > For example, the first test in the patch:
> > > org/apache/harmony/auth/tests/module/Krb5LoginModuleTest.java
> > >
> > > was originally excluded in build.xml with the following  
comment:
> > > <!-- The test is under development     -->
> > > <!-- excluded to avoid random failures -->
> > >
> > > Since that the code of the test was not modified, so the  
test is still
> > > incomplete and should not be returned back.
> > >
> > > Other tests may have similar problems too
> > >
> > > Thanks,
> > > Mikhail
> > >
> > > 2007/2/19, tatyana doubtsova <[EMAIL PROTECTED]>:
> > > > Hi,
> > > > I've run excluded classlib tests on win ia32 drlvm|ibmvme,  
linux ia32
> > > > drlvm|ibmvme, linux em64t drlvm and found out that 12  
tests from
> > common
> > > > x-lists passed on all 5 configurations. The tests are from  
auth, luni,
> > > > nio_char, rmi, sound, text, lang-manegement, and security  
modules.
> > I've
> > > > prepared the patch for x-lists clean-up:
> > > > https://issues.apache.org/jira/browse/HARMONY-3194.   
Commiters, would
> > you,
> > > > please, have a look and apply, if there are no objections?



--
Alexei Zakharov,
Intel ESSD