-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Great, I will try it out--
Marlon On 4/13/12 7:43 AM, Jasha Joachimsthal wrote: > Jasha Joachimsthal > > Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466 > US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free) > > www.onehippo.com > > > On 12 April 2012 03:59, Franklin, Matthew B. <[email protected]> wrote: > >>> -----Original Message----- >>> From: Marlon Pierce [mailto:[email protected]] >>> Sent: Tuesday, April 10, 2012 12:01 PM >>> To: [email protected] >>> Subject: Re: Quality assurance steps for Rave >>> > I started a checklist of tests at > http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as you > have seen from the [Rave Wiki] messages. >>> >>> Good set of tests. I had been going through most of these on my own >>> before each release, but that will become impossible to keep up with as the >>> functionality grows. >>> > > * Assuming the user interface tests are done manually, updating the wiki > every time would be cruddy. Is there good test/assurance process > management software out there? Should we do this with an online (Google) > spreadsheet? >>> >>> I say create a blocking Pre-release test task in Jira and have subtasks >>> for each "section" of the tests. (we can use copy so it isn't such a manual >>> task) >>> > > * Do we want instead to actively maintain selenium tests? I started this >>> but > let it drop because these only worked in Firefox, and Firefox's frequent > updates broke the tests. The tests themselves would have to be actively > maintained--if you change or add a user interface feature, you need to >>> also > update the test. >>> >>> I think if we develop the simplest possible tests to protect against >>> regression, it could work without too much effort. We need to make it an >>> explicit part of the process though. It has also been my experience that >>> recorded test cases don't work without some modification. I have also had >>> decent success in keeping code-based (not selenese) tests working.... >>> > > * Are there other recommendations beside Selenium? >>> > >> I've just created a test project [1] with JBehave [2] which uses Selenium >> under the hood. > >> Scenario: User creates a new account and logs in into the portal >> When I go to "http://localhost:8080/portal"; >> Then I see the login page >> When I follow the new account link >> Then I get the new account form >> When I fill in the form with username "newuser" password "password" >> confirmpassword "password" email "[email protected]" >> And I submit the new account form >> Then I see the login page >> And A message appears "Account successfully created" >> When I fill in the login form with username "newuser" password "password" >> Then I see my portal page with the add new widgets box > > >> you can run it with mvn clean install or run the NewUserStories class from >> your IDE. >> It does not clean up the newly created user so you cannot run it twice >> successfully without cleaning up the user. > >> [1] https://github.com/jashaj/PortalTests >> [2] http://jbehave.org/ > >> > > > Marlon > > > On 4/6/12 3:32 PM, Raminderjeet Singh wrote: >>>>> +1 with Marlon on having a checklist of features. I know going forward >>> this > list can grow and we may need more people to verify the build. >>>>> >>>>> Based on my experience doing 0.10 release, Matt and others have done a > great job putting all the steps together into scripts. Thanks! >>>>> >>>>> After the code freeze announcement, Release manager can tag the current > code and verification can be done on that particular tag. One developer > (release manager also) can verify the code tag for the feature list and >>> if very > thing looks good can do the release based on the tag. Matt can comment > more as i have to still understand when the pom versions are updated. >>>>> >>>>> Thanks >>>>> Raminder >>>>> >>>>> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote: >>>>> >>>>> I'd like to propose the following. >>>>> >>>>> * Develop a list of Rave features that should be tested, put this on >>> the wiki, > and update it every time a new feature is added. I'm happy to get this > started. Recommendations for tools to automate some of this are welcome. >>>>> >>>>> >>>>> * Regular testing of the above feature list (not just in preparation for > releases). We would need a record of this (who tested, when, etc). If >>> we do > this manually, we could keep records by updating the wiki, posting to the >>> dev > list, etc. Recommendations for better tools are of course welcome. >>>>> >>>>> >>>>> * All features should be tested before release by at least one person. >>> The > Release Manager should coordinate this. We could continue to do this as >>> part > of the current Release Candidate process, but it may be better to have an > intermediate SVN tag that can go through QA before we start an official >>> vote. > This would allow us to keep the trunk open for commits while we test and > avoid canceled releases. >>>>> >>>>> >>>>> I'm not a QA expert, so comments welcome. >>>>> >>>>> >>>>> >>>>> Marlon >>>>> >>>>> >> > -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPiBmZAAoJEEfVXEODPFIDLmsIAJhErl0l1Ryyv5RJSCaa6vt9 VvlII/FpaRjYAlAr/NRnvUhgRrY78YSBjKsJqapTOLEbLDqkbBSozo1VPoESPX21 T1ztNQQlhn572PjLSlvhwWHFDWh/esNGZ1Zga8g2lTEo8kfNfPVqZVue3cLi5b1N lxK+4aOqTR2L57w8xg4q5DmclxwwKR62RkVR8POW+kUF2efp5GyIbqCXy/dAmnrj 9uGU71gUexN5+cRueKvfUUa5FVXLEtJPnuLr9o8HJsLXDxokmqL+8VVZdoDoFbbR b6RI7KUIfSZVA6gWMXuQrOwRn1r4D1ZZIHMHO+8ZIgW3Xauk2CSYpSPlESdwhGo= =3M7b -----END PGP SIGNATURE-----
