On Aug 26, 2009, at 3:46 AM, angela wrote: > Hi Eric, > > Thanks for your time reviewing the test plan. My reply in line. > > Eric J. Ray ??: >> >> Comments: >> >> I'm not comfortable with the lack of _full_ test automation, >> including >> verification of >> the final installed image. If the final answer is that we must have >> manual testing, >> I think QE will need to own that. > There are some scenarios that couldn't be done in automated test, such > as we plan to include some user case which is about > interoperability, we > will install other OS (Windows, Linux) and install Solaris to see > whether it can be installed alongside other OS's. >
I assume that the final test spec will be set up in such a way that all of the required testing for enhancements/bug fixes will be fully automated, with optional manual steps that QE can run. Right? > As for the verification of the final installation image, actually we > have script to do that. It relies on STEP framework to control the > test > system to do reboot and post verify. This will be automated after > integrated into PIT. Currently we can only manually do that. > > We re-arranged the manual test execution, so the test schedule will be > changed to: > 11/23-12/01 automated test on virtual machines, manual test on x86 > and sparc > OK > Do you have any other concerns on these? >> >> I think we should have PIT runs. > Yes, the automated test suite will be integrated into PIT. > We plan to have 2 phases: > In phase 1, we will integrate the automated test suite into STC at the > same time when project integration according to the new policy. > And in phase 2, we will integrate the suite into PIT. What's the time difference between the first and the second? You'll own execution until the second, right? How will you provide development engineers the ability to manually execute the automated tests, particularly the ones that require STEP? >> >> Why no memory leak testing? > This has been discussed in our test meeting with Sue. Sue agreed that, > the text installer will use python, no need to do memory leak test. >> >> I don't see where it says to verify that the installed image >> works...does it? > I mentioned it in section Test Program Summary. I'll write more > clearly > in next version. > > Later I'll send out the new version incorporated with comments I > received. > > Thanks, > Angela
