Hi again, Thanks everyone for your feedback on these questions. As is evident from what most people are saying, the "correct" answer is that tests should be run by default, and we should only have "correct" tests which require the correct behaviour. This can then work into the future with a TDD (test driven development) methodology where all new code should be accompanied with valid test classes.
However, I think we need to be careful in drawing a distinction between the "ideal / correct" answer, and the position that we are currently in. The DSpace codebase has code that is over 8 years old, and this is our first foray into providing automated tests. We must recognise that the environment that we are working in is not the pure TDD environment that we may strive for. Therefore I would caution against trying to move to the "ideal" situation immediately. Whilst we should have a plan of how to achieve the ideal, perhaps we should move to it in a phased approach over the next release or two? For example, as it stands, if we follow the "correct" route, then until someone finds enough time (probably a week or two of effort) to fix up DCDate, then we'll be in a position where 'mvn package' will fail. Because we're in a position where all users need to run this command, it will doubtless cause us support issues. What I'd prefer is that we perhaps launch with the un-ideal state (tests off by default, and DCDate with a broken test). If we have the effort available we can setup a CI server to run and report on the tests. Of course, as developers we can ensure that we run the tests. If tests for a class such as DCDate fail, it will at least highlight any new errors being introduced. Then perhaps once we have released 1.7, once DCDate is fixed, and once we have more test coverage, we can re-visit the decision and consider turning on tests by default. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: +64 (0)9 373 7599 x81928 On 1/08/2010, at 3:13 PM, Stuart Lewis wrote: > Hi all, > > At last week's DSpace Development meeting it was decided that the GSoC > testing project is appropriate to be committed into 'trunk' so that it can > become part of DSpace version 1.7 which is due out before the end of the > year. As the mentor for the project, I work with the Pere Villega (the > student) to get the project ready for inclusion. > > The project is currently looking very good, and runs a lot of junit tests > across the code. The majority of the tests that have been created so far are > relate to the browse, content, and storage parts of the core API. The tests > are executed via maven. > > Before we merge Pere's branch into trunk, I'd like to ask for some opinions > from the community at large: > > --- > i) Should the tests run automatically every time the code is packaged? > - Maven tries to run tests when the 'mvn package' command is given. > Whilst this is a good thing to do (as it highlights any errors that are > present in that build), it does add an extra minute to the build time. This > doubles the time it takes to run 'mvn package'. There are two choices of > what to do here: > 1) Make the tests run automatically, and if you want to skip them, run > 'mvn -Dmaven.test.skip=true package' > 2) Make the tests optional, to run them type 'mvn > -Dmaven.test.skip=false package' > --- > > --- > ii) Some parts of DSpace are 'broken' right now. A good example is DCDate. It > has some issues, such as if the date is the 1st Jan, then it assumes the > granularity of the date is just a year. This is because if you just set a > year, it defaults to the 1st Jan. However, if you want a date of the 1st > Jan, it won't work. Therefore some of the tests for DCDate fail. Some of > the tests that have failed in other classes have been fixed as a result of > the testing. This is a good 'win' for the project as it is highlighting > errors for us already. However some classes such as DCDate need a lot of > work, and are already being reviewed by some members of the community. The > question is: > - Should we commit 'correct' tests that fail for DCDate, or should we > commit tests that are broken in the same way as DCDate is broken? Whilst it > is probably not good practice to commit incorrect tests, having tests that > are going to fail for a while (until DCDate is fixed) causes problems as it > means that builds will always fail. This is a side effect of introducing a > testing framework around buggy code, rather than having the framework come > first. My preference would be to commit the faulty test class, but to use a > Test Driven Development methodology for the re-development of DCDate to > ensure we get it right next time. > --- > > If you have an opinion about either of these, please express it so that we > can come to a consensus. > > Finally, a few people have had a look over Pere's code, but it would be great > for more people to check it before we merge it into trunk. It is very easy > to try: > > - svn co http://scm.dspace.org/trac/dspace/browser/sandbox/gsoc/2010/testing . > - mvn -Dmaven.test.skip=false package > > You should see the code compile, then an in-memory database will start up, > and some tests will be run. If you want to look at the tests, look in > dspace-test/src/test/java/org/dspace/ > > Many thanks, > > > Stuart Lewis > IT Innovations Analyst and Developer > Te Tumu Herenga The University of Auckland Library > Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand > Ph: +64 (0)9 373 7599 x81928 ------------------------------------------------------------------------------ The Palm PDK Hot Apps Program offers developers who use the Plug-In Development Kit to bring their C/C++ apps to Palm for a share of $1 Million in cash or HP Products. Visit us here for more details: http://p.sf.net/sfu/dev2dev-palm _______________________________________________ Dspace-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dspace-devel
