Hi Sebastian, Some time ago I've created a test plan for Vasiliy where I tried to cover all the cases and which was revieved by Alexei and Maxim.
It's currently on Russian, and probably is incomplete in some aspects. I'll try to do translate it on English and share with others ASAP (hope, today later or tomorrow). Vasiliy will be responsible for the reports about the tests execution. . Any yes, our fault is that we did not share it with the community from the very beginning :(. You are right about the automated junit tests. I'll try to help Maxim with this :) Best regards, Irina. On Sun, Mar 24, 2013 at 11:15 AM, seba.wag...@gmail.com < seba.wag...@gmail.com> wrote: > I would be already happy if we do the following: > > 1) Enable the Junit test to run automated (by using the Backup Import via > JUnit as example) > So that every committer can add new JUnit tests that run with every Nightly > Build. > > 2) Start a list of test/use cases that should be performed with any > release. > Maybe there is already such a list ? What did Alexey, Artyom, Irina, Vasya, > Yuliya test at all ? > How did they manage the work of "testing", did they agree on any tests that > need to be performed ? > > Sebastian > > > > 2013/3/24 Maxim Solodovnik <solomax...@gmail.com> > > > I guess we need to improve+enlarge our automated tests and rely on it in > > the future. > > Right now it is necessary to run manual tests :( > > I'll try to write 1-2 tests per day/week (too much work right now :(( ) > > > > > > On Sun, Mar 24, 2013 at 11:52 AM, seba.wag...@gmail.com < > > seba.wag...@gmail.com> wrote: > > > > > Wicket will help to do tests. However our client is 100% Flash now. > > > > > > Do we want to run UI tests in the Flash UI or do we want to only run > > JUnit > > > tests automated ? > > > > > > How can we define which JUnit tests run automated ? > > > > > > From my perspective the more we can test automated the less time you > > spend > > > on even more painful tasks. > > > Cause every test that is _not_ automated means that: > > > - It is likely that nobody will do testing > > > - A extremly painful process will start where we maintain a wiki > > document > > > that lists all tests (with all problems including, like nobody takes > care > > > of those documents, nobody can really control if those tests have been > > > performed at all or not, et cetera) > > > > > > So from my perspective putting some time into an automated test is > still > > > much less pain then trying now to re-run all those manual tests, mail > > ping > > > pong and discussion with every release that we do. > > > > > > Sebastian > > > > > > > > > > > > 2013/3/24 Maxim Solodovnik <solomax...@gmail.com> > > > > > > > It is hard for me to answer such long letters :))) > > > > > > > > >> Yeah, well how should any user do a test if there is no public > demo > > > > I'm not sure what is the status of demo.dataved.ru, it allows "self > > > > registration" and it is up 24/7, but you are right, there were no > "Call > > > to > > > > test". But I was sure My emails like "people I'm going to release, > > please > > > > stop me if it is too early" is sort of call to test it and let me > > > know.... > > > > > > > > I agree on "automated testing", I promise I'll add out tests to the > > build > > > > (I forgot about it, will create JIRA issue). > > > > > > > > since we will be on Wicket we can finally start writing tests on our > UI > > > > similar to their tests (never tried that) > > > > I do like automated tests, it is just not my favorite task :) > > > > > > > > according to our (and Apache guide) > > > > http://openmeetings.apache.org/ReleaseGuide.html > > > > " > > > > > > > > *Before voting +1 PMC members are required to download the signed > > source > > > > code package, compile it as provided, and test the resulting > executable > > > on > > > > their own platform, along with also verifying that the package meets > > the > > > > requirements of the ASF policy on releases.* > > > > > > > > " > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Mar 24, 2013 at 11:17 AM, seba.wag...@gmail.com < > > > > seba.wag...@gmail.com> wrote: > > > > > > > > > We did extensive testing of 2.1 (Alexey, Artyom, Irina, Vasya, > > Yuliya) > > > > > => where did they perform the tests? I thought we would invite the > > > > > community to help us testing. > > > > > > > > > > *1) there were no issues reported by users* > > > > > Yeah, well how should any user do a test if there is no public > demo? > > I > > > > also > > > > > did not hear any call on the user mailing list that users are > invited > > > to > > > > > test. > > > > > *2) We better release 2.1.1 or 2.2 in a month than wait another 6 > > > months* > > > > > I agree on that. But our past agreement was more like "dev complete > > => > > > > > release". That model will not work for our future. > > > > > And I want to make sure that everybody involved understands that. > > > > > > > > > > IMHO our lack of automated testing and the need for a manual test / > > > click > > > > > through of all the features is one of the biggest issues in our > > current > > > > > project. > > > > > For example I do not understand why the JUnit test for the backup > > > import > > > > > was never integrated into the Nightly builds? I mean all that work > > that > > > > > you've put into that. Simply nobody uses it now. > > > > > It would be such a nice thing to wake up every morning and see what > > > test > > > > > fails and what to look at? I guess there are only a couple of bits > > > > missing > > > > > to get the backup import running automated but I don't understand > > what > > > > > keeps us away from doing that? > > > > > > > > > > Similar for the rest of the Junit tests. Of couse a good amount of > > the > > > > > tests are just outdated. > > > > > But if there would be at least a minimal subset of tests that run > > > > > automated, that would be an improment by 100%, cause at the moment, > > > just > > > > > zero tests run automated. > > > > > This will become even more interesting with Wicket, where you can > > test > > > a > > > > > lot of the UI stuff with simple JUnit tests. > > > > > The manual work that Alexey, Artyom, Irina, Vasya, Yuliya and > anybody > > > > else > > > > > involved has done for 2.1 > > > > > => Will need to happen with every release. 2.1.1, 2.2, ... > > > > > An approach like "A feature that has been tested in the release 2.1 > > > needs > > > > > no more testing in a release 2.1.1 (or 2.2)". I will not agree on > > that > > > in > > > > > any sense. Every release does need a full test. > > > > > And IMHO this approach will not scale at all with the growing > number > > of > > > > > committers. > > > > > > > > > > It would be great if we start thinking about what we will do to > > improve > > > > > that in the future? > > > > > > > > > > The tools are basically there but it seems like nobody involved in > > the > > > > > project believes that automated tests make sense (except me) ? > > > > > > > > > > From @Alexey I know that he believes only additions to the feature > > add > > > > > value to the end product. And it seems like "testing" is not a > > > "feature" > > > > > that adds any value to the end user from that perspective. > > > > > So my questions would be: Do we really want to do the same amount > of > > > > manual > > > > > click-through tests that we do now with every release ?! > > > > > I mean: Am I the only person sick of downloading every release and > > > > clicking > > > > > through every feature 30 minutes to give a "+1" ?! > > > > > > > > > > Sebastian > > > > > > > > > > > > > > > 2013/3/24 Maxim Solodovnik <solomax...@gmail.com> > > > > > > > > > > > We did extensive testing of 2.1 (Alexey, Artyom, Irina, Vasya, > > > Yuliya) > > > > > > additional causes are: > > > > > > 1) there were no issues reported by users > > > > > > 2) We better release 2.1.1 or 2.2 in a month than wait another 6 > > > months > > > > > > > > > > > > ps Apach Wicket has 1 month release cycle .... I believe we > should > > > have > > > > > 2-3 > > > > > > month > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Mar 24, 2013 at 10:20 AM, seba.wag...@gmail.com < > > > > > > seba.wag...@gmail.com> wrote: > > > > > > > > > > > > > Hi Maxim, > > > > > > > > > > > > > > I was wondering if the testing phase that I thought we have > > agreed > > > on > > > > > > > already happen? > > > > > > > Or is there another reason why you initiated this RC? > > > > > > > > > > > > > > Sebastian > > > > > > > > > > > > > > > > > > > > > 2013/3/23 Maxim Solodovnik <solomax...@gmail.com> > > > > > > > > > > > > > > > Dear OpenMeetings Community, > > > > > > > > > > > > > > > > I would like to start a vote about releasing Apache > > OpenMeetings > > > > > 2.1.0 > > > > > > > RC3 > > > > > > > > > > > > > > > > RC2 was rejected due to broken audio/video setup panel > > > > > > > > > > > > > > > > Main changes are covered in the Readme: > > > > > > > > > > > > > > > > > > http://svn.apache.org/repos/asf/openmeetings/tags/2.1RC3/README > > > > > > > > > > > > > > > > Full Changelog: > > > > > > > > > > > http://svn.apache.org/repos/asf/openmeetings/tags/2.1RC3/CHANGELOG > > > > > > > > > > > > > > > > Release artefacts: > > > > > > > > https://dist.apache.org/repos/dist/dev/openmeetings/2.1/rc3/ > > > > > > > > > > > > > > > > Tag: > http://svn.apache.org/repos/asf/openmeetings/tags/2.1RC3/ > > > > > > > > > > > > > > > > PGP release keys (signed using C467526E): > > > > > > > > > > https://dist.apache.org/repos/dist/dev/openmeetings/2.1/rc3/KEYS > > > > > > > > > > > > > > > > Vote will be open for 72 hours. > > > > > > > > > > > > > > > > [ ] +1 approve > > > > > > > > [ ] +0 no opinion > > > > > > > > [ ] -1 disapprove (and reason why) > > > > > > > > > > > > > > > > My vote is +1. > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > WBR > > > > > > > > Maxim aka solomax > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Sebastian Wagner > > > > > > > https://twitter.com/#!/dead_lock > > > > > > > http://www.webbase-design.de > > > > > > > http://www.wagner-sebastian.com > > > > > > > seba.wag...@gmail.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > WBR > > > > > > Maxim aka solomax > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Sebastian Wagner > > > > > https://twitter.com/#!/dead_lock > > > > > http://www.webbase-design.de > > > > > http://www.wagner-sebastian.com > > > > > seba.wag...@gmail.com > > > > > > > > > > > > > > > > > > > > > -- > > > > WBR > > > > Maxim aka solomax > > > > > > > > > > > > > > > > -- > > > Sebastian Wagner > > > https://twitter.com/#!/dead_lock > > > http://www.webbase-design.de > > > http://www.wagner-sebastian.com > > > seba.wag...@gmail.com > > > > > > > > > > > -- > > WBR > > Maxim aka solomax > > > > > > -- > Sebastian Wagner > https://twitter.com/#!/dead_lock > http://www.webbase-design.de > http://www.wagner-sebastian.com > seba.wag...@gmail.com >
