Thanks Amila for weighing in. Comments inline: On Dec 16, 2013, at 11:29 AM, Amila Jayasekara <thejaka.am...@gmail.com> wrote:
> Hi Suresh, > > I have some comments inline. > > > On Mon, Dec 16, 2013 at 10:53 AM, Suresh Marru <sma...@apache.org> wrote: > Hi All, > > This is a very good question. Lets discuss these options so we are consistent > across releases. > > If we look at the way we are doing releases, we are calling a feature freeze > and code freeze and cutting a release. Most of the time, our build is broken. > Jenkins statistics for Airavata is not looking good at all [1]. > > There is something wrong with the Jenkins configurations. I tried to figure > out sometime back I was unable to do so. Even though builds are successful in > our local machines they are failing intermittently in Jenkins. > > We are barely fixing the build a day before the release, putting out an RC > and testing on it and releasing it in a quick succession. > > This is not entirely true. For the past few months I only experienced one or > two build breaks (maybe less). I build couple of times per week. I believe > usually build is stable and with integration tests passing, we always get a > workable version. I know its not a good practice not to rely on the build > server. But commiters have personal discipline to keep the build stable. > Nevertheless we must fix Jenkins configuration issue. May be we should put focus on Jenkins configuration? Any volunteers? > > As we are seeing on user lists, we have users upgrading with every release. I > think we should increase the release quality. > > +1 for this. > > I would vote for atleast 3 RC’s per release. If we are not finding issues in > first RC, I would say, either the software has magically become too too good > or we are not doing through testing. I suspect the later. > > I guess you mentioned this under assumption that build is not stable. Half of my assumption is on Jenkins, so if builds are ok and Jenkins is thinking wrong, then we can alleviate it by fixing it. > I will propose the following, please counter it and lets agree on a process: > > * Lets post a RC1 as is (which means it will have a snapshot). This pack, we > should all test as much as possible, so its more of a test candidate then a > release candidate. If it helps, we can use the name TC1. I am not particular > on the naming but trying to emphasize the need for having atleast more RC's > per release. > > I am not sure whether we really need a TC. The release manager should be > doing some verifications on the RC before putting it out. Therefore it should > be a RC. Anyhow i am fine having TC concept and trying it out. We probably should stick to RC, but I think the onus should not be on the RM to test it. They should coordinate and mobilize every one to do the testing including doing a testing bit more than others. But my point is, we should test and the only way to do that is to put a series of RC’s and have focused testing. Suresh > > What we really need is set of verifiable test cases. > > Thank you > Regards > Amila > > > * If we do not expose significant issues in RC/TC 1 then we proceed with RC2 > which will follow the proper release process. But if we have a reasonable > issues bought out, we need a RC2/TC2 also without following the release > process. > > * The key thing I am proposing is, we keep doing RC/TC’s until we all are > sure the quality is good enough with documented known issues. When we are > sure, then we proceed to have RC with proper release process. > > So this will mean more testing and twice (or more) the times every one has to > test, but I think it is worth it. This might also get over the 6 week release > cycle, but I think we need to trade for some quality releases as we march > towards 1.0. > > Suresh > [1] - https://builds.apache.org/job/Apache%20Airavata/ > > > On Dec 15, 2013, at 4:28 PM, Lahiru Gunathilake <glah...@gmail.com> wrote: > > > > > Hi Chathuri, > > > > I think having snapshot as the version in RC is wrong. Every RC has to be > > like a release and if it pass we just call a vote/discussion thread and do > > the release. If we do with snapshot and if things go right, then have to > > change versions and test again. But we can do the release just by changing > > snapshot without testing but that wrong AFAIT. > > > > I remember doing this mistake in earlier release with RC1 build. I think we > > can stick to the release management instructions in airavata.org. > > > > Regards > > Lahiru > > > > > > On Fri, Dec 13, 2013 at 3:43 PM, Chathuri Wimalasena <kamalas...@gmail.com> > > wrote: > > Hi All, > > > > Airavata 0.11 RC1[1] is ready for testing. > > > > Here are some pointers for testing > > • Verify the fixed issue for this release [2] > > • Verify the basic workflow composition/execution/monitoring > > scenarios from > > • Airavata 5 & 10 min tutorials [3],[4] > > • Verify airavata client samples > > • Verify the stability with derby & mysql backend databases > > • Verify that the XBaya JNLP distribution works > > • Verify deploying Airavata server in a tomcat distribution > > Please report any issues[5] if you encounter while testing. Thank you for > > your time in validating the release. > > > > Regards, > > Chathuri (On behalf of Airavata PMC) > > > > [1] https://dist.apache.org/repos/dist/dev/airavata/0.11/RC1/ > > [2] > > https://issues.apache.org/jira/browse/AIRAVATA-278?jql=project%20%3D%20AIRAVATA%20AND%20fixVersion%20%3D%20%220.11%22%20ORDER%20BY%20status%20DESC%2C%20priority%20DESC > > [3] > > http://airavata.apache.org/documentation/tutorials/airavata-in-5-minutes.html > > [4] > > http://airavata.apache.org/documentation/tutorials/airavata-in-10-minutes.html > > [5] https://issues.apache.org/jira/browse/AIRAVATA > > > > > > > > > > > > > > -- > > System Analyst Programmer > > PTI Lab > > Indiana University > >
