Hi George, On 1/27/06, George Harley <[EMAIL PROTECTED]> wrote: > Sure. And it should be the role of the test framework to inform users > about which tests got run, passed, failed, got skipped etc.
see below > > It would not disturb most of the people because the test will pass in 'bad' > > environment. But those, who know about these tests will sometimes grep > > logs to validate configuration. > > > Why use logs to tell us information that the unit test framework can already > provide ? Who wants to have to grep through log files when the test runner > can provide us with what we need ? Sorry but I did not quite understand. I do not see anything related to the 'skipped' status in the junit.framework.TestResult If there is such a status then it would be excellent, it is the option #1 in the first mail in the thread Thanks, Mikhail > > > Thanks, > > Mikhail > > > > > > > >> Alternatively, they could be > >> included as part of a general test suite but be purposely skipped over at > >> test execution time using a > >> test exclusion list understood by the test runner. > >> > >> > >> Best regards, > >> George > >> ________________________________________ > >> George C. Harley > >> > >> > >> > >> > >> > >> Tim Ellison <[EMAIL PROTECTED]> > >> 27/01/2006 08:53 > >> Please respond to > >> harmony-dev@incubator.apache.org > >> > >> > >> To > >> harmony-dev@incubator.apache.org > >> cc > >> > >> Subject > >> Re: [testing] code for exotic configurations > >> > >> > >> > >> > >> > >> > >> Anton Avtamonov wrote: > >> > >>>> Note that I could create my own provider and test with it, but what I > >>>> > >> would > >> > >>>> really want is to test how my EncryptedPrivateKeyInfo works with > >>>> AlgorithmParameters from real provider as well as how my other classes > >>>> > >> work > >> > >>>> with real implementations of crypto Engines. > >>>> > >>>> Thanks, > >>>> Mikhail. > >>>> > >>>> > >>> Hi Mikhail, > >>> There are 'system' and 'unit' tests. Traditionally, unit tests are of > >>> developer-level. Each unit test is intended to test just a limited > >>> piece of functionality separately from other sub-systems (test for one > >>> fucntion, test for one class, etc). Such tests must create a desired > >>> environment over the testing fucntionality and run the scenario in the > >>> predefined conditions. Unit tests usually able to cover all scenarios > >>> (execution paths) for the tested parts of fucntionality. > >>> > >>> What are you talking about looks like 'system' testing. Such tests > >>> usually run on the real environment and test the most often scenarious > >>> (the reduntant set, all scenarios usually cannot be covered). Such > >>> testing is not concentrated on the particular fucntionality, but > >>> covers the work of the whole system. > >>> A sample is: "run some demo application on some particular platform, > >>> with some particular providers installed and perform some operations". > >>> > >>> I think currently we should focus on 'unit' test approach since it is > >>> more applicable during the development (so my advise is to revert your > >>> tests to install 'test' providers with the desired behavior as George > >>> proposed). > >>> However we should think about 'system' scenarios which can be run on > >>> the later stage and act as 'verification' of proper work of the entire > >>> system. > >>> > >> I agree with all this. The unit tests are one style of test for > >> establishing the correctness of the code. As you point out the unit > >> tests typically require a well-defined environment in which to run, and > >> it becomes a judgment-call as to whether a particular test's > >> environmental requirements are 'reasonable' or not. > >> > >> For example, you can reasonably expect all developers to have an > >> environment to run unit tests that has enough RAM and a writable disk > >> etc. such that if those things do not exist the tests will simply fail. > >> However, you may decide it is unreasonable to expect the environment to > >> include a populated LDAP server, or a carefully configured RMI server. > >> If you were to call that environment unreasonable then testing JNDI and > >> RMI would likely involve mock objects etc. to get good unit tests. > >> > >> Of course, as you point out, once you are passing the unit tests you > >> also need the 'system' tests to ensure the code works in a real > >> environment. Usage scenarios based on the bigger system are good, as is > >> running the bigger system's test suite on our runtime. > >> > >> Regards, > >> Tim > >> > >> > >> > >>> -- > >>> Anton Avtamonov, > >>> Intel Middleware Products Division > >>> > >>> > >> -- > >> > >> Tim Ellison ([EMAIL PROTECTED]) > >> IBM Java technology centre, UK. > >> > >> > >> > >> > > > > > >
