Hi, Andrew.
I very much agree with your complaint and analysis.
One of the things I will be proposing and implementing for 0.12 is a new
top-level entry point for qpid tests. The idea is that this would be the
one place to go to get an up or down result on a change. Our current
situation results in each person doing his or her testing in a somewhat
different way.
Fwiw, I also heartily endorse using Apache's CI tools, and thanks for
volunteering to drive it.
Justin
On Wed, 9 Mar 2011, Andrew Kennedy wrote:
Guys,
I appreciate we are in a rush to get 0.10 ready to ship, however this is no
excuse for letting testing slide.
I have seen several instances of changes that have caused multiple
regressions under various test profiles, and this sort of thing will only
increase as the number of combinations of profiles increases. Additionally,
some commits have even prevented the project from compiling. I have done both
of these things in the past myself, particularly with regards to the C++ test
profiles, and this is one reason I spent so much time validating and testing
my network code.
Can we all please make an effort to run at least the Java 0-8 and 0-10 and
the C++ profiles before committing changes, to give other developers
confidence that any new test failures are caused by *their* code, not
somebody else's?
One of my colleagues is working on increased unit test coverage via the
introduction of mock objects, which will reduce our dependency on system
tests and allow regressions to be found when testing individual components. I
agree that this is the way forward, as our test coverage is far too low
currently. However, I could not find out any solid metrics on coverage to
provide as an example. I notice that although there is a Qpid instance on the
Nemo server at Sonar (see link below) it does not give coverage metrics,
although there is a *lot* of useful information there. Does anyone know how
this could be fixed, or who in the project has access to configure this?
http://nemo.sonarsource.org/dashboard/index/274568
Additionally, I would be happy to set up and manage a CI build for Qpid on
the ASF Hudson service. I believe the PMC chair just needs to grant access
(see first link below) to the correct group. This would allow us to have a
shared view of build and test statuses (see second link below) and more. I
know many of us run our own private CI instances, but I think having a shared
resource everyone in the project can refer to would be very useful.
http://wiki.apache.org/general/Hudson
https://builds.apache.org/hudson/
I can raise a Qpid JIRA for this to track progress, if people approve of the
idea?
Cheers,
Andrew.
--
-- andrew d kennedy ? do not fold, bend, spindle, or mutilate ;
-- http://grkvlt.blogspot.com/ ? edinburgh : +44 7582 293 255 ;
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
