>-----Original Message----- >From: Jasha Joachimsthal [mailto:[email protected]] >Sent: Friday, April 13, 2012 7:43 AM >To: [email protected] >Subject: Re: Quality assurance steps for Rave > >Jasha Joachimsthal > >Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466 >US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free) > >www.onehippo.com > > >On 12 April 2012 03:59, Franklin, Matthew B. <[email protected]> wrote: > >> >-----Original Message----- >> >From: Marlon Pierce [mailto:[email protected]] >> >Sent: Tuesday, April 10, 2012 12:01 PM >> >To: [email protected] >> >Subject: Re: Quality assurance steps for Rave >> > >> >-----BEGIN PGP SIGNED MESSAGE----- >> >Hash: SHA1 >> > >> >I started a checklist of tests at >> >http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as >you >> >have seen from the [Rave Wiki] messages. >> >> Good set of tests. I had been going through most of these on my own >> before each release, but that will become impossible to keep up with as the >> functionality grows. >> >> > >> >* Assuming the user interface tests are done manually, updating the wiki >> >every time would be cruddy. Is there good test/assurance process >> >management software out there? Should we do this with an online >(Google) >> >spreadsheet? >> >> I say create a blocking Pre-release test task in Jira and have subtasks >> for each "section" of the tests. (we can use copy so it isn't such a manual >> task) >> >> > >> >* Do we want instead to actively maintain selenium tests? I started this >> but >> >let it drop because these only worked in Firefox, and Firefox's frequent >> >updates broke the tests. The tests themselves would have to be actively >> >maintained--if you change or add a user interface feature, you need to >> also >> >update the test. >> >> I think if we develop the simplest possible tests to protect against >> regression, it could work without too much effort. We need to make it an >> explicit part of the process though. It has also been my experience that >> recorded test cases don't work without some modification. I have also had >> decent success in keeping code-based (not selenese) tests working.... >> >> > >> >* Are there other recommendations beside Selenium? >> > >I've just created a test project [1] with JBehave [2] which uses Selenium >under the hood.
+1 for JBehave > >Scenario: User creates a new account and logs in into the portal >When I go to "http://localhost:8080/portal"; >Then I see the login page >When I follow the new account link >Then I get the new account form >When I fill in the form with username "newuser" password "password" >confirmpassword "password" email "[email protected]" >And I submit the new account form >Then I see the login page >And A message appears "Account successfully created" >When I fill in the login form with username "newuser" password "password" >Then I see my portal page with the add new widgets box > > >you can run it with mvn clean install or run the NewUserStories class from >your IDE. I would think we would want to do this from a different profile, right? >It does not clean up the newly created user so you cannot run it twice >successfully without cleaning up the user. > >[1] https://github.com/jashaj/PortalTests >[2] http://jbehave.org/ > > > >> > >> >Marlon >> > >> > >> >On 4/6/12 3:32 PM, Raminderjeet Singh wrote: >> >> +1 with Marlon on having a checklist of features. I know going forward >> this >> >list can grow and we may need more people to verify the build. >> >> >> >> Based on my experience doing 0.10 release, Matt and others have done >a >> >great job putting all the steps together into scripts. Thanks! >> >> >> >> After the code freeze announcement, Release manager can tag the >current >> >code and verification can be done on that particular tag. One developer >> >(release manager also) can verify the code tag for the feature list and >> if very >> >thing looks good can do the release based on the tag. Matt can comment >> >more as i have to still understand when the pom versions are updated. >> >> >> >> Thanks >> >> Raminder >> >> >> >> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote: >> >> >> >> I'd like to propose the following. >> >> >> >> * Develop a list of Rave features that should be tested, put this on >> the wiki, >> >and update it every time a new feature is added. I'm happy to get this >> >started. Recommendations for tools to automate some of this are >welcome. >> >> >> >> >> >> * Regular testing of the above feature list (not just in preparation for >> >releases). We would need a record of this (who tested, when, etc). If >> we do >> >this manually, we could keep records by updating the wiki, posting to the >> dev >> >list, etc. Recommendations for better tools are of course welcome. >> >> >> >> >> >> * All features should be tested before release by at least one person. >> The >> >Release Manager should coordinate this. We could continue to do this as >> part >> >of the current Release Candidate process, but it may be better to have an >> >intermediate SVN tag that can go through QA before we start an official >> vote. >> >This would allow us to keep the trunk open for commits while we test and >> >avoid canceled releases. >> >> >> >> >> >> I'm not a QA expert, so comments welcome. >> >> >> >> >> >> >> >> Marlon >> >> >> >> >> >-----BEGIN PGP SIGNATURE----- >> >Version: GnuPG/MacGPG2 v2.0.16 (Darwin) >> >Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ >> > >> >>iQEcBAEBAgAGBQJPhFlJAAoJEEfVXEODPFIDPZ4H/2FM9G6b6mJuJgkong4cA >W >> >wg >> >>TMj8Vks5ykOsRspkqIff4Q8ItzkVrrq0hKb+1p2gaaGj8nG6/BhKnh+CaSsuwfhC >> >>QqeucJdj4/0RreCx+eQ9M0A17Y9O21yZxKuKQaiPHHWbv32NZWHsuycUPDP >+ >> >MmvI >> >>mC5/w9+N2jQf2O7y6X0fR6ydfyggJrhBbM0PNY3yp6O7dHb91KqUz4Vm5giXl1 >> >H/ >> >>OzG0NEDv/EweIbTCeEJWOs2oDZGZgkRCT7z0LoJXXWNHifUSSJMuUDwPu0 >M >> >5CIIJ >> >>tX16D/JdC2ik9FtarCHeQV101uEJYrC3Mx4F9Lb/PO0Hj+ePOBrha9rMRnSOcK0 >= >> >=06BC >> >-----END PGP SIGNATURE----- >>
