On Mon, Dec 16, 2013 at 9:28 AM, Suresh Marru <sma...@apache.org> wrote:
> Thanks Amila for weighing in. Comments inline: > > On Dec 16, 2013, at 11:29 AM, Amila Jayasekara <thejaka.am...@gmail.com> > wrote: > > > Hi Suresh, > > > > I have some comments inline. > > > > > > On Mon, Dec 16, 2013 at 10:53 AM, Suresh Marru <sma...@apache.org> > wrote: > > Hi All, > > > > This is a very good question. Lets discuss these options so we are > consistent across releases. > > > > If we look at the way we are doing releases, we are calling a feature > freeze and code freeze and cutting a release. Most of the time, our build > is broken. Jenkins statistics for Airavata is not looking good at all [1]. > > > > There is something wrong with the Jenkins configurations. I tried to > figure out sometime back I was unable to do so. Even though builds are > successful in our local machines they are failing intermittently in Jenkins. > > > > We are barely fixing the build a day before the release, putting out an > RC and testing on it and releasing it in a quick succession. > > > > This is not entirely true. For the past few months I only experienced > one or two build breaks (maybe less). I build couple of times per week. I > believe usually build is stable and with integration tests passing, we > always get a workable version. I know its not a good practice not to rely > on the build server. But commiters have personal discipline to keep the > build stable. Nevertheless we must fix Jenkins configuration issue. > > May be we should put focus on Jenkins configuration? Any volunteers? > > > > > As we are seeing on user lists, we have users upgrading with every > release. I think we should increase the release quality. > > > > +1 for this. > > > > I would vote for atleast 3 RC’s per release. If we are not finding > issues in first RC, I would say, either the software has magically become > too too good or we are not doing through testing. I suspect the later. > How about we keep a checklist of release tests? I know we already send a mail on dev on what needs to be tested for each RC, but I need that is too abstract. For core developers of Airavata I think there should be test cases predefined (a test document if you may). Since we have several core developers in the list we can atleast decide upon what must be tested and make sure that each test case is covered by atleast one developer for a RC. > > > > I guess you mentioned this under assumption that build is not stable. > > Half of my assumption is on Jenkins, so if builds are ok and Jenkins is > thinking wrong, then we can alleviate it by fixing it. > > > I will propose the following, please counter it and lets agree on a > process: > > > > * Lets post a RC1 as is (which means it will have a snapshot). This > pack, we should all test as much as possible, so its more of a test > candidate then a release candidate. If it helps, we can use the name TC1. I > am not particular on the naming but trying to emphasize the need for having > atleast more RC's per release. > > > > I am not sure whether we really need a TC. The release manager should be > doing some verifications on the RC before putting it out. Therefore it > should be a RC. Anyhow i am fine having TC concept and trying it out. > > We probably should stick to RC, but I think the onus should not be on the > RM to test it. They should coordinate and mobilize every one to do the > testing including doing a testing bit more than others. But my point is, we > should test and the only way to do that is to put a series of RC’s and have > focused testing. > A TC should be something internal IMO. But when we are going for a release it should be alpha, beta and then RC releases. I think it need not be mandatory for the RMs to do pre-evaluation of the builds other than making sure all the unit tests and integration tests pass. Once an RC is confirmed of release quality I think we can follow the actual release cycle from the trunk itself with since its in a code freeze anyway. > > Suresh > > > > > What we really need is set of verifiable test cases. > > > > Thank you > > Regards > > Amila > > > > > > * If we do not expose significant issues in RC/TC 1 then we proceed with > RC2 which will follow the proper release process. But if we have a > reasonable issues bought out, we need a RC2/TC2 also without following the > release process. > > > > * The key thing I am proposing is, we keep doing RC/TC’s until we all > are sure the quality is good enough with documented known issues. When we > are sure, then we proceed to have RC with proper release process. > > > > So this will mean more testing and twice (or more) the times every one > has to test, but I think it is worth it. This might also get over the 6 > week release cycle, but I think we need to trade for some quality releases > as we march towards 1.0. > > > > Suresh > > [1] - https://builds.apache.org/job/Apache%20Airavata/ > > > > > > On Dec 15, 2013, at 4:28 PM, Lahiru Gunathilake <glah...@gmail.com> > wrote: > > > > > > > > Hi Chathuri, > > > > > > I think having snapshot as the version in RC is wrong. Every RC has to > be like a release and if it pass we just call a vote/discussion thread and > do the release. If we do with snapshot and if things go right, then have > to change versions and test again. But we can do the release just by > changing snapshot without testing but that wrong AFAIT. > > > > > > I remember doing this mistake in earlier release with RC1 build. I > think we can stick to the release management instructions in airavata.org. > > > > > > Regards > > > Lahiru > > > > > > > > > On Fri, Dec 13, 2013 at 3:43 PM, Chathuri Wimalasena < > kamalas...@gmail.com> wrote: > > > Hi All, > > > > > > Airavata 0.11 RC1[1] is ready for testing. > > > > > > Here are some pointers for testing > > > • Verify the fixed issue for this release [2] > > > • Verify the basic workflow composition/execution/monitoring > scenarios from > > > • Airavata 5 & 10 min tutorials [3],[4] > > > • Verify airavata client samples > > > • Verify the stability with derby & mysql backend databases > > > • Verify that the XBaya JNLP distribution works > > > • Verify deploying Airavata server in a tomcat distribution > > > Please report any issues[5] if you encounter while testing. Thank you > for your time in validating the release. > > > > > > Regards, > > > Chathuri (On behalf of Airavata PMC) > > > > > > [1] https://dist.apache.org/repos/dist/dev/airavata/0.11/RC1/ > > > [2] > https://issues.apache.org/jira/browse/AIRAVATA-278?jql=project%20%3D%20AIRAVATA%20AND%20fixVersion%20%3D%20%220.11%22%20ORDER%20BY%20status%20DESC%2C%20priority%20DESC > > > [3] > http://airavata.apache.org/documentation/tutorials/airavata-in-5-minutes.html > > > [4] > http://airavata.apache.org/documentation/tutorials/airavata-in-10-minutes.html > > > [5] https://issues.apache.org/jira/browse/AIRAVATA > > > > > > > > > > > > > > > > > > > > > -- > > > System Analyst Programmer > > > PTI Lab > > > Indiana University > > > > > >
