Awesome! Thanks a for those notes Devaraj!
Very useful for the unfortunates who did not got the chance to join the meeting ... Le 19 févr. 2013 20:27, "Devaraj Das" <[email protected]> a écrit : > - Stabilizing 0.96 / CI > -- not running continuously > -- Roman: is it a good idea to run IT under Bigtop > - the HBase unit tests are good.. > - What about HBase as a backend for hive - tests for those > - Do we think there is value with the integration with the rest > of the hadoop ecosystem > -- Jon: What's needed to make this work > -- Roman: Collective agreement that we want to solve this > -- Stack: We need to run the IT tests from Enis and Sergey running > continuously > -- Roman: If we all agree that this is something needs to be fixed > .. then yes we would talk about mechanics. Bigtop doesn't have > expertise in HBase and hence HBase folks would have to debug failures. > -- Roman: Bigtop is committed to tests. Less than a dozen tests for > hbase currently.. > -- We have been running validation around RC time. Find all kinds of > issues - sometimes trivial (maybe a config issue), > -- Roman can offer CI for trunk but will work for only hadoop-2 line. > -- Roman does first line of triage. > -- No issues to do with other ecosystem artifacts. Bigtop ensures > the right artifacts are in place. > -- hadoop-2 is important but not particular about the version of > hadoop within 2. For example 2.0.2 > -- Gary: Will be good to run security tests > -- Roman & Devaraj to talk on how this can be done/implemented > > On 0.96 branching > - Lars: > -- We will have three branches to maintain. > - Stack: we need to stabilize quickly > - Enis: What about 0.95. > - Stack: Just do the snapshots thing. Every week, give a snapshot > - Enis: we've done a bunch of stuff in RPC.. If we have to break > something, we can if it is beta (0.96-beta). > - Agreement is there generally ... Debate on the name with snapshot > versus 95.0/96.0 > -- 0.95 experimental .. 0.96 will be stable > -- If we go with 0.95, releasing will be easier as well.. > -- 0.95 will not be in production.. > -- 0.96 will be off 0.95 branch and not trunk based > > - How do we go about committing issues.. (issue commit rate is low) > -- 0.94 is stable - 2 new commits and 2 new bugs a day > -- 0.96 has lots of issues not reviewed > -- Break up the patch into multiple smaller pieces to make review easier > -- Branching on a big feature was suggested - > --- Issues: committer needs to be there > Sergey: If the rate of change is high on common code (to the > branches) then merging will be tough > --- Jon: Refactor should be done in the main branch (since it > doesn't add any new funtionality) > --- Release often to reduce #backports overall and issues with that.. > > - Review process .. how to drive a review to closure. Effort goes > waste if the review process is not completed. The same reviewer should > continue to review the patch .. > - Hard to enforce any process > - Enis: there should be a summary of the patch and all that .. so that > the review process is helped.. Hard to understand the architecture of > the patch unless documented > - Jon: It should be easy to make a one-to-one correspondence between > the description and the patch > - The commit should have only the jira# as opposed to pages of description > > - Component owners: is this working. Committers need to be forthcoming > with reviews > -- Maybe review the modules and add some more if needed. > > -- Good that we have more contributions coming than we have > reviewers, but unless we keep up, we will plateau > -- Mail on dev@ list if review doesn't happen > > - Dev co-ordination: > -- How best can we pull together > -- Priorities: > --- Getting 0.96 out is priority > --- Backports to 0.94 will happen .. until 0.96 is stable > --- 0.92 release ? Any committer who wants to make a release can do > so (maybe with some backports, etc.) > --- Backporting can be tough if there are bugs and the bugfix has > to be applied to all branches > > - HBASE-2600 - this requires a change in the client and the server. > They have to be changed in lock-step. Its hard to do this .. Jon > doesn't want to have the fix for 0.96. So 0.98 might be another > singular release. Maybe do a rewrite of the meta after taking a lock > on the meta, do a shim layer to handle the backward compat. between > 0.96 and 0.98 > > - What do people want to get into 0.94 > -- The biggest thing - Snapshots, mostly new code, about a 3rd of the > stuff in 0.94 already > -- Compactions improvments - no backport > > - How devs can better co-ordinate > -- Snapshots co-ordination working well > -- One page design is useful (makes it readable and all) > -- How about handling the stripe compaction - where an idea leads to > a bunch of others > -- Again write-up should be done > > - Should we change the description to match the comments > -- Two ideas suggested: > --- We probably should have the description updated with the > "Date: new description" if the issue at hand is updated > --- Should we have a summary after a bunch of comments - yes > > - The face-to-face meetings are useful. We should semd out the minutes > of the discussion to the dev list. We probably should have more > focused huddles. Discuss but don't decide (decide on the jira)! > > - Jon: Would people be amenable to merge sooner rather than later on > snapshots? Tested and being beaten up. > - Stack: Yes > > - What else goes in in 0.96: > -- RPC refactor > -- ROOT removal > -- Compaction stuff > -- Package name mailing list thread - there is now a jira on that. > We shouldn't break clients. Package name changes is not worth the > trouble. > > - A bunch of discussions on the RPC with KeyValue/Cells > > - What do we do about usability > -- It'll be nice if we don't need to change configs.. > -- Maybe expose more metrics and then allow for online config > changes since automatic config is difficult and needs to be battle > tested and all. > > - Benchmarking of the release: > -- We should measure the overhead of PB stuff >
