George Harley wrote:
Hi Geir,
From my point of view the answer is yes, I ran the tests for the
patched module. The build/test server picked up the downstream breakages
in the other modules. I fixed matters so that the builds ran again.
Right - I thought we considered the build/test server a safety to keep
accidents from getting away from us, but given the delay and the
distribution of people throughout the planet working on this (rather
than all sitting in the same office), we want to avoid breakages at all
costs. (Again, I'm guilty of the same thing...)
Elsewhere in this thread I am trying to cooperate with others to make it
simpler to run more - just the necessary - tests prior to making a commit.
I guess the question is "what is necessary"? Given a dependency chain
A <- B <- C
I don't think it's only the immediate dependencies of a given module A,
but rather the full dependency graph, because a "1 away" module (C)
might use a direct dependency (B) in a way different than the test cases
for that direct dependency do (especially since we aren't quite up to
speed with doing implementation testing...)
Thank you for your support.
Um...
geir
Best regards,
George
Geir Magnusson Jr wrote:
So the answer is "no"?
George Harley wrote:
Geir Magnusson Jr wrote:
Just curious (and this isn't a criticism - I'm just as guilty of not
doing this)...
Don't you run the tests before committing?
Hi Geir,
Depends what do you mean by "the tests".
The change was completely encapsulated in the text component. I ran
the text tests. I checked that the new tests introduced in the issue
patch gave the same results when run against a 5.0 RI. All was fine.
The automated build and test server (which runs all the tests in all
the modules) found the problem. That is it's job and it did it well.
The problem was found relatively quickly and has now been remedied.
The turnaround time would have been quicker if the automated build
and test server issued a failure notice on failing tests but that is
now in the process of getting sorted out.
IMHO this problem was a bit of an odd problem as the failures did not
arise in the patched module's test suite but in other modules. That
is a good indication that the test suite for text is in need of some
scrutiny. That is a useful thing to be made aware of.
Best regards,
George
geir
George Harley wrote:
Hi,
It *seems* like things started failing after I committed the
changes for HARMONY-205 last night. I'm looking into this now. If
the investigation begins to take up too much time I will back the
changes out.
Best regards,
George
Stepan Mishura wrote:
The same for me + DatagramChannelTest
Thanks,
Stepan.
On 4/11/06, Mark Hindess wrote:
No. These:
F
org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
E
org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
E
org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
E
org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
F java.security.cert.X509CertSelectorTest.testMatch
F java.security.cert.X509CertSelectorTest.testClone
F tests.api.java.sql.DateTest.testSetTimelong
F tests.api.java.sql.DateTest.testToString
F tests.api.java.sql.DateTest.testValueOf
F tests.api.java.sql.TimestampTest.testSetNanosint
F tests.api.java.sql.TimestampTest.testToString
F tests.api.java.util.DateTest.test_toGMTString
F tests.api.java.util.DateTest.test_toString
Regards,
Mark.
On 4/11/06, Stepan Mishura <[EMAIL PROTECTED]> wrote:
On 4/11/06, Mark Hindess wrote:
Yes. I was using the failureproperty mechanism. Trying to get this
property propogated back to the top level ant file was what I was
having trouble with.
Using a file as you suggest might help. I'll give that a try
shortly...
Incidentally, I'm seeing 12 failures and 3 errors on r393111.
I guess that you have 9 tests from DatagramChannelTest passed.
And 12 +
3 =
15 :-)
(And
there are typos - "mathc" should be "match" - in the failure
messages
for java.security.cert testMatch and testClone.)
I've fixed typo.
Thanks,
Stepan.
Regards,
Mark.
On 4/11/06, Stepan Mishura <[EMAIL PROTECTED]> wrote:
On 4/11/06, Mark Hindess wrote:
Stepan,
I have another build running (but without notifications going to
the
list) that runs:
1) build (with reference jdk)
2) build with what we created with 1)
3) test
4) create classlists and compare with class load data for
applications
(tomcat, geronimo, continuum, etc.)
5) generate JAPI results
I'd like to fail this build at step 3, but I can't see an
easy way
to
get 'ant -f make/build.xml test' to run all tests and then
fail if
any
of the module test sub-targets had test failures. I could parse
the
output I suppose, but I'd rather get ant to propagate the junit
tasks
failure property back up to the top level. I've tried a
couple of
things and none seem to work. Any suggestions welcome.
Mark, did you try failureproperty parameter for junit task?
We may add it to ant sub-targets to raise a flag, for example,
create
file "
TESTS.FAILED" in the root, when tests for some module fail.
And in
the
end
of tests suite run check whether this file exists on not.
Thanks,
Stepan.
Regards,
Mark.
On 4/11/06, Stepan Mishura <[EMAIL PROTECTED]> wrote:
Hi,
I've checked out at morning last updates, built the code base
and
run
the
tests …and there are 24 tests failures!
There are 9 tests failures in
org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
–
I
saw
these
failures before from time to time. It seems that tests
depend on
some
race
conditions because they may pass if I rerun them. Paulex, are
these
tests
passing for you?
And there are new 15 test failures. So now if I'll make a code
update
or a
bug fix how I can be 100% sure that I don't do any regression?
Currently if a commit breaks the Harmony classlib build a
notification
with
subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
be
send
to
harmony-commits mailing list. Is it possible to have the same
for
tests?
I
mean that after completing automatic build the existing
classlib
tests
suite
should be run. If there are failing tests then a report
notification
with
corresponding subject will be send.
Thanks,
Stepan Mishura
Intel Middleware Products Division
--
Mark Hindess <[EMAIL PROTECTED]>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail:
[EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
--
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail:
[EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
Thanks,
Stepan Mishura
Intel Middleware Products Division
--
Mark Hindess <[EMAIL PROTECTED]>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail:
[EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
--
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail:
[EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
Thanks,
Stepan Mishura
Intel Middleware Products Division
--
Mark Hindess <[EMAIL PROTECTED]>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
--
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
Thanks,
Stepan Mishura
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]