On 9/17/07, Vemund Ostgaard <[EMAIL PROTECTED]> wrote: > I'm interested in running the junit tests with a remote server, but as > far as I can see there is no easy way to do so yet. > > I found two relevant jiras: > https://issues.apache.org/jira/browse/DERBY-1973 (Support running JUnit > tests directly with a remote server) > http://issues.apache.org/jira/browse/DERBY-2638 (Create an option for > junit tests to run only client tests) > > As these are currently unassigned I was thinking of taking a closer look > myself. > > A couple of questions: > > What interface should be used to instruct the test framework that you > want to use a network server (probably started manually) running on a > given host & portnumber, instead of the default localhost location? And > along the same lines, what is the best way to instruct the test > framework that you want to run only the "client tests" (or only the > "embedded tests")? > > For choosing to run only a subset of tests, like the "client tests", I > see a few possibilities: > * Create some new top level suite, for instance "Suites.client" that > runs only the "client tests". > * Set a property (on the commandline) when starting Suites.All (or any > other subsuite) that instructs the framework to only run the "client tests". > * Make it possible to do both of the above. > > For instructing the framework to use a specific hostname and portnumber > instead of the defaults I don't have any good ideas except using > properties. The DERBY-1973 report says "Ideally this would be through a > test decorator and not setting properties on the command line.", though. > Any suggestions on the best way to solve this? The TestConfiguration > class seems to read system properties for framework (embedded or > client), hostname and port, but is this code there only to make the > tests run under the old harness and intended to be removed when all the > tests are converted? > > Is it likely that all "client tests" will work with a remote server, or > will some tests be written in a way that requires the server(s) to run > on the same host as the test framework? Perhaps some tests have to start > the networkserver as part of the test, and because of this it has to run > on the same host. In this case it would actually be three groups of > tests: embedded, local networkserver and remote networkserver. > > If anyone has given all of this some thought I'd be happy to hear. > > Vemund >
Coming from attempting to implement remote server testing in the 'old' test harness, I have some limited input, and little to offer in the way of solutions... Starting to reason from the tests, I think part of the issue is that there are tests that test the correct starting and shutting down of the network server; or that need the server to be started with specific properties as opposed to the 'normal' mode. And as far as I know we cannot start and shutdown a server on a remote machine, and thus these tests cannot run with a remote server. Theoretically, all tests that do not need to test starting/shutting down, or that do not test the server with specific properties set, should, be 'runnable' with remote server that is already running. However, is that really necessary? What things could go wrong when connecting to a remote server versus a server on localhost? I would think mostly things to do with connections, permissions, import/export. So, maybe it makes sense after all to have a dedicated set of tests for remote server testing. Note that with remote server testing, trouble brews in the area of cleanup; I believe the current junit framework attempts to delete files that get created by the server; and I think deleting of files on a remote machine should be impossible. I wasted considerable time with the old test harness trying to identify tests that would run cleanly in sequence; only to find that a previously working set was made to fail because a uncooperative test/testcase was added. It may be better to create a suite of tests, normally part of suites.All, which can/should be run against a remote server as well. For the desire to be able to run all client tests; that is more an efficiency issue useful for someone working on client only changes; when one is certain there is no need to run the tests with embedded. I think maybe a class that travels through the test suites and identifies the ones that run with DerbyNetClient would cover that requirement. The converse could also be true (run only embedded tests), but I believe this can be achieved by excluding derbyclient.jar from the classpath. I have a hard time to imagine how to arrange for a configurable port number and host without using properties in some manner. But I imagine some default values, and/or a scheme to identify existing servers running could be devised. Maybe someone else has good suggestions. Myrna
