Thanks for correcting me. Yes, integration tests it is. And yes, we can't do anything serious without them.
On Thu, Jul 22, 2010 at 14:01, Assaf Arkin <[email protected]> wrote: > On Wed, Jul 21, 2010 at 10:09 PM, Antoine Toulme > <[email protected]>wrote: > >> There actually will be crucial differences between specs and functional >> tests. Mainly specs are a free for all to mock, squeeze, open wide open >> internals to check the code as close as possible. >> >> Functional tests should run with the stock buildr, no changes. >> >> Your points are valid: we still need to run tests in a safe environment, >> where we make sure to dispose of artifacts created. >> So definitely a before and an after are in order. >> The two tests added are peculiar - they don't create files just yet. >> >> However you think we're better off writing the Buildfile every time ? In >> rake I see they kept the Buildfile in their own folders. This way I can >> picture myself pointing people to "real" examples, real files they can see >> and try for themselves to reproduce problems. >> >> Even if we keep the Buildfiles, we can still copy them dynamically in a >> testbed. So your idea stands sound to me. >> >> At this point, I'd like to commit what I have, and refine to do a testbed >> as >> you suggest. Believe it or not, this is looking pretty exciting to me :) >> > > I think you're looking at integration tests here. > > At the granularity of a Buildfile and running a full build (how else can > you test packaging), you're going to be testing multiple things in one go. > > Functional tests look at one function at a time. And you want to quickly > run a small set of them to test the code as you're writing it (TDD), so > functional tests contain a lot of mocking and stubbing. > > A lot of problems don't surface until you run end-to-end integration tests. > You definitely want directory for each test, with the initial set of files. > Besides being much easier to maintain, and like you said, can point people > there to look at actual examples. > > Assaf > > >> >> On Wed, Jul 21, 2010 at 21:48, Peter Donald <[email protected]> >> wrote: >> >> > Hi, >> > >> > On Thu, Jul 22, 2010 at 2:45 AM, Antoine Toulme < >> [email protected]> >> > wrote: >> > > I'm looking for volunteers to take a quick look at the patch attached >> to: >> > > https://issues.apache.org/jira/browse/BUILDR-456 >> > > >> > > And tell me what they think. >> > >> > As a first cut it looks not bad. I would expect to see a library of >> > assertions built up over time to make it easy to check the output and >> > results of the builds. >> > >> > One thing I would like to see is a consistency between the spec tests >> > and the functional tests in how they setup the environment. In both >> > cases you are typically going to need to delete then create a scratch >> > directory structure at start of test, configure (and populate) the >> > maven repository. Both tests also may create some dirs and setup a >> > build file (although in the spec tests it is done in memory while in >> > the functional tests you are likely to be copying around files). At >> > the end of the tests the scratch directory should be nuked. >> > >> > So extracting this into a common api would be great. It would also >> > mean that it could be more easily used by outside people that are >> > writing buildr plugins. I could imagine an api like >> > >> > # Put in test_helper/sepc_helper >> > Buildr::Test.scratch_directory = File.basedir(__FILE__) + "/../tmp" >> > Buildr::Test.clean_scratch_on_destroy = ENV["TEST_CLEAN_SCRATCH"].nil? >> > ? true : ENV["TEST_CLEAN_SCRATCH"] == 'true' >> > >> > # called from setup >> > Buildr::Test.init # >> > >> > # Called to copy artifact from local maven repo to maven repo in test >> > Buildr::Test.install_artifact 'com.biz:my-widget:jar:2.0' >> > Buildr::Test.install_artifact 'com.biz:my-other-widget:jar:2.0' >> > >> > # Example code to setup a functional test >> > FileUtils.cp_r File.basedir(__FILE__) + "/../functional/" + >> > test_name, Buildr::Test.project_directory >> > >> > # called from teardown >> > Buildr::Test.destroy >> > >> > Thoughts? >> > >> > -- >> > Cheers, >> > >> > Peter Donald >> > >> > >
