On 13 April 2012 13:52, Franklin, Matthew B. <[email protected]> wrote:
> >-----Original Message----- > >From: Jasha Joachimsthal [mailto:[email protected]] > >Sent: Friday, April 13, 2012 7:43 AM > >To: [email protected] > >Subject: Re: Quality assurance steps for Rave > > > >Jasha Joachimsthal > > > >Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466 > >US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll > free) > > > >www.onehippo.com > > > > > >On 12 April 2012 03:59, Franklin, Matthew B. <[email protected]> wrote: > > > >> >-----Original Message----- > >> >From: Marlon Pierce [mailto:[email protected]] > >> >Sent: Tuesday, April 10, 2012 12:01 PM > >> >To: [email protected] > >> >Subject: Re: Quality assurance steps for Rave > >> > > >> >-----BEGIN PGP SIGNED MESSAGE----- > >> >Hash: SHA1 > >> > > >> >I started a checklist of tests at > >> >http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as > >you > >> >have seen from the [Rave Wiki] messages. > >> > >> Good set of tests. I had been going through most of these on my own > >> before each release, but that will become impossible to keep up with as > the > >> functionality grows. > >> > >> > > >> >* Assuming the user interface tests are done manually, updating the > wiki > >> >every time would be cruddy. Is there good test/assurance process > >> >management software out there? Should we do this with an online > >(Google) > >> >spreadsheet? > >> > >> I say create a blocking Pre-release test task in Jira and have subtasks > >> for each "section" of the tests. (we can use copy so it isn't such a > manual > >> task) > >> > >> > > >> >* Do we want instead to actively maintain selenium tests? I started > this > >> but > >> >let it drop because these only worked in Firefox, and Firefox's > frequent > >> >updates broke the tests. The tests themselves would have to be > actively > >> >maintained--if you change or add a user interface feature, you need to > >> also > >> >update the test. > >> > >> I think if we develop the simplest possible tests to protect against > >> regression, it could work without too much effort. We need to make it > an > >> explicit part of the process though. It has also been my experience > that > >> recorded test cases don't work without some modification. I have also > had > >> decent success in keeping code-based (not selenese) tests working.... > >> > >> > > >> >* Are there other recommendations beside Selenium? > >> > > > >I've just created a test project [1] with JBehave [2] which uses Selenium > >under the hood. > > +1 for JBehave > > > > >Scenario: User creates a new account and logs in into the portal > >When I go to "http://localhost:8080/portal"; > >Then I see the login page > >When I follow the new account link > >Then I get the new account form > >When I fill in the form with username "newuser" password "password" > >confirmpassword "password" email "[email protected]" > >And I submit the new account form > >Then I see the login page > >And A message appears "Account successfully created" > >When I fill in the login form with username "newuser" password "password" > >Then I see my portal page with the add new widgets box > > > > > >you can run it with mvn clean install or run the NewUserStories class from > >your IDE. > > I would think we would want to do this from a different profile, right? > We need to think about the right setup of the integration tests. It would be good if an experienced tester could help with that. I just wanted to show an option how to test the portal without clicking through it manually. > > >It does not clean up the newly created user so you cannot run it twice > >successfully without cleaning up the user. > > > >[1] https://github.com/jashaj/PortalTests > >[2] http://jbehave.org/ > > > > > > >> > > >> >Marlon > >> > > >> > > >> >On 4/6/12 3:32 PM, Raminderjeet Singh wrote: > >> >> +1 with Marlon on having a checklist of features. I know going > forward > >> this > >> >list can grow and we may need more people to verify the build. > >> >> > >> >> Based on my experience doing 0.10 release, Matt and others have done > >a > >> >great job putting all the steps together into scripts. Thanks! > >> >> > >> >> After the code freeze announcement, Release manager can tag the > >current > >> >code and verification can be done on that particular tag. One developer > >> >(release manager also) can verify the code tag for the feature list and > >> if very > >> >thing looks good can do the release based on the tag. Matt can comment > >> >more as i have to still understand when the pom versions are updated. > >> >> > >> >> Thanks > >> >> Raminder > >> >> > >> >> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote: > >> >> > >> >> I'd like to propose the following. > >> >> > >> >> * Develop a list of Rave features that should be tested, put this on > >> the wiki, > >> >and update it every time a new feature is added. I'm happy to get this > >> >started. Recommendations for tools to automate some of this are > >welcome. > >> >> > >> >> > >> >> * Regular testing of the above feature list (not just in preparation > for > >> >releases). We would need a record of this (who tested, when, etc). If > >> we do > >> >this manually, we could keep records by updating the wiki, posting to > the > >> dev > >> >list, etc. Recommendations for better tools are of course welcome. > >> >> > >> >> > >> >> * All features should be tested before release by at least one > person. > >> The > >> >Release Manager should coordinate this. We could continue to do this as > >> part > >> >of the current Release Candidate process, but it may be better to have > an > >> >intermediate SVN tag that can go through QA before we start an official > >> vote. > >> >This would allow us to keep the trunk open for commits while we test > and > >> >avoid canceled releases. > >> >> > >> >> > >> >> I'm not a QA expert, so comments welcome. > >> >> > >> >> > >> >> > >> >> Marlon > >> >> > >> >> > >> >-----BEGIN PGP SIGNATURE----- > >> >Version: GnuPG/MacGPG2 v2.0.16 (Darwin) > >> >Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > >> > > >> > >>iQEcBAEBAgAGBQJPhFlJAAoJEEfVXEODPFIDPZ4H/2FM9G6b6mJuJgkong4cA > >W > >> >wg > >> > >>TMj8Vks5ykOsRspkqIff4Q8ItzkVrrq0hKb+1p2gaaGj8nG6/BhKnh+CaSsuwfhC > >> > >>QqeucJdj4/0RreCx+eQ9M0A17Y9O21yZxKuKQaiPHHWbv32NZWHsuycUPDP > >+ > >> >MmvI > >> > >>mC5/w9+N2jQf2O7y6X0fR6ydfyggJrhBbM0PNY3yp6O7dHb91KqUz4Vm5giXl1 > >> >H/ > >> > >>OzG0NEDv/EweIbTCeEJWOs2oDZGZgkRCT7z0LoJXXWNHifUSSJMuUDwPu0 > >M > >> >5CIIJ > >> > >>tX16D/JdC2ik9FtarCHeQV101uEJYrC3Mx4F9Lb/PO0Hj+ePOBrha9rMRnSOcK0 > >= > >> >=06BC > >> >-----END PGP SIGNATURE----- > >> >
