zk-less assignment will be off by default. co-location of meta and master is on by default now. We can turn it off if needed.
Thanks, Jimmy On Thu, Jun 12, 2014 at 2:12 PM, Mikhail Antonov <olorinb...@gmail.com> wrote: > So how would the defaults and dependencies look amongst these 3, once > committed to 0.99? > > - co-location of meta and master > - zk-less assignment > - timeline-consistent replicas? > > As I understand - > - All 3 are off by default > - zk-less assignments and region replicas to not be turned on at the same > time > - zk-less assignments require meta co-location > - region replicas don't care about meta co-location? > > -Mikhail > > > 2014-06-12 14:07 GMT-07:00 Jonathan Hsieh <j...@cloudera.com>: > > > sgtm. > > > > > > On Mon, Jun 9, 2014 at 10:46 AM, Enis Söztutar <enis....@gmail.com> > wrote: > > > > > Thanks Jon. > > > > > > I think we'll try the rebase approach first. I'll start the effort > today > > > and see how far along I can get with that. Rebasing each patch might > be a > > > bit more work actually. If this turns out painful, we'll revert to the > > > merge approach similar to what you describe. > > > > > > Enis > > > > > > > > > On Sat, Jun 7, 2014 at 7:41 AM, Jonathan Hsieh <j...@cloudera.com> > wrote: > > > > > > > On Fri, Jun 6, 2014 at 5:05 PM, Enis Söztutar <enis....@gmail.com> > > > wrote: > > > > > > > > > My preference would be to do the rebase-then-merge style (last > style > > in > > > > > your comment). For each patch, I am hoping for all the changes > > between > > > > > committed version and rebased version to be mechanical. I like > > having a > > > > > linear history with explicit commit-to-patch mapping. > > > > > > > > > > If the changes are of a mechanical nature and that each patch > > compiles > > > > and > > > > passes unit tests along the way, then rebase-then-merge is perfectly > > fine > > > > by me. > > > > > > > > Even though snapshots were fairly orthogonal to the rest of the > > codebase, > > > > when we did merges we had problems maintaining the compiles and > passes > > > with > > > > every commit invariants when we tried rebase-then-merge approach. > > After > > > > each rebase we would end up doing a bunch of work and still end up > > > failing > > > > unit tests. In that case we (jesse, matteo, myself) went to the > merge > > > > approach. We actually merged into the snapshot branch a few times > > > fixing > > > > things due to changes in trunk that broke parts of the snapshots > > branch. > > > > Here's a more accurate picture of what the commit history looked like > > in > > > > git (we dev'ed in git and in end had to recommit this all in svn). > > > > > > > > m (essentially empty merge patch). > > > > |\ > > > > t | (trunk patch with no impact) > > > > | s6* (patch to fix newly introduced problem) > > > > |/| > > > > t | > > > > t*| (trunk patch that broke snapshots again) > > > > | s5* (branch bug fixes due to merge) > > > > | s4* (mechanical fixups due to the merge) > > > > |/| > > > > t | > > > > t | > > > > t | > > > > | s3 > > > > | s2 > > > > | s1 > > > > |/ > > > > t > > > > t > > > > > > > > This approach was captured in github and had the added benefit of > > > > preserving exact history, and having known good points preserving the > > > > compile/unit test invariants on trunk. (unfortunately, some of this > > > > history was lost when we ported the git history over to svn, but that > > > isn't > > > > a problem any more). If you want to see the whole thing, look at > my > > > git > > > > repo: > > > > > > > > git remote add jmhsieh g...@github.com:jmhsieh/hbase.git > > > > git log --oneline --graph --color jmhsieh/hbase-7290 > > > > > > > > > > > > Can we do history-preserving merge with the first style? I can do the > > > > > rebases and upload interdiff if that is big so that we can compare > > on a > > > > > per-patch basis. > > > > > > > > > > > > > > It is possible. > > > > > > > > > > > > > > > > > > > > > > > > > Enis > > > > > > > > > > > > > > > On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <j...@cloudera.com> > > > wrote: > > > > > > > > > > > When we merged snapshots branch in we did this: > > > > > > > > > > > > t = trunk commit > > > > > > s = snapshot branch commit > > > > > > m = merge point. > > > > > > > > > > > > During work: > > > > > > t > > > > > > t > > > > > > t > > > > > > | s3 > > > > > > | s2 > > > > > > | s1 > > > > > > |/ > > > > > > t > > > > > > t > > > > > > > > > > > > During after merge: > > > > > > m (essentially empty merge patch). > > > > > > t \ > > > > > > t | > > > > > > t | > > > > > > | s4* (fixups due to the merge) > > > > > > | s3 > > > > > > | s2 > > > > > > | s1 > > > > > > |/ > > > > > > t > > > > > > t > > > > > > > > > > > > Does your proposal mean you are going to do this instead? > > > > > > > > > > > > s3* (modified due to rebase) > > > > > > s2* (modified due to rebase) > > > > > > s1 > > > > > > t > > > > > > t > > > > > > t > > > > > > | s (now dead hbase-10070 branch) > > > > > > | s > > > > > > | s > > > > > > | s > > > > > > |/ > > > > > > t > > > > > > t > > > > > > > > > > > > If it is the then we should probably have a review of the > modified > > > > > patches. > > > > > > If it the same as the snapshot merge, then we just need a review > > of > > > > the > > > > > > merge point delta (as well as a full review). > > > > > > > > > > > > Personally I prefer the merge for these large feature branches -- > > it > > > > > > guarantees that each commit is compilable, and reflects what you > > guys > > > > > have > > > > > > been testing for a while. If you go with the last approach you > > might > > > > > have > > > > > > stuff broken, and in the mainline commit path. > > > > > > > > > > > > Jon. > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell < > > apurt...@apache.org > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <e...@apache.org > > > > > > wrote: > > > > > > > > > > > > > > > This VOTE is for merging back the remaining changes in branch > > to > > > > > trunk. > > > > > > > If > > > > > > > > passes, we will rebase the branch on top of current trunk, in > > > which > > > > > we > > > > > > > will > > > > > > > > keep the commit-per-issue log history. After that we will do > a > > > git > > > > > > merge > > > > > > > > for the branch keeping the history clean and not squashing > the > > > > > > commits. I > > > > > > > > expect rebasing to be straightforward, however with some > manual > > > > > > conflict > > > > > > > > resolution. After the merge we'll keep running the tests to > > make > > > > sure > > > > > > > > everything is ok. > > > > > > > > > > > > > > > > > > > > > > Just to clarify that would look something like this: > > > > > > > > > > > > > > $ git checkout HBASE-10070 > > > > > > > $ git rebase --ignore-date master > > > > > > > (fixups, git add, git rebase --continue, etc, etc, etc) > > > > > > > $ git checkout master > > > > > > > $ git merge HBASE-10070 > > > > > > > > > > > > > > ? > > > > > > > > > > > > > > That sounds good to me, the final merge should be a fast > forward > > > > merge. > > > > > > > > > > > > > > Use of ' --ignore-date' could be mildly controversial. It's not > > > > > strictly > > > > > > > necessary because the commits for 10070 will appear grouped in > > > > history, > > > > > > but > > > > > > > then dates on commits will be discontiguous in that section of > > > > > history. I > > > > > > > suggest using that option so the order of commits and dates > sort > > > the > > > > > same > > > > > > > on master. > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <e...@apache.org > > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > Last week we started some discussion[4] for merging branch > > > > > > hbase-10070[1] > > > > > > > > into trunk. It seems like the consensus there is to do the > > merge > > > > > sooner > > > > > > > > rather than later. > > > > > > > > > > > > > > > > > > > > > > > > We had branched hbase-10070 in Feb out of trunk[5]. The > branch > > > > > contains > > > > > > > 55 > > > > > > > > jiras committed[2]. Out of these 55, 15 has already been > > > committed > > > > to > > > > > > > trunk > > > > > > > > and backported to hbase-10070 branch[3]. > > > > > > > > > > > > > > > > This VOTE is for merging back the remaining changes in branch > > to > > > > > trunk. > > > > > > > If > > > > > > > > passes, we will rebase the branch on top of current trunk, in > > > which > > > > > we > > > > > > > will > > > > > > > > keep the commit-per-issue log history. After that we will do > a > > > git > > > > > > merge > > > > > > > > for the branch keeping the history clean and not squashing > the > > > > > > commits. I > > > > > > > > expect rebasing to be straightforward, however with some > manual > > > > > > conflict > > > > > > > > resolution. After the merge we'll keep running the tests to > > make > > > > sure > > > > > > > > everything is ok. > > > > > > > > > > > > > > > > An overview of the changes, and the status of the work can be > > > found > > > > > > under > > > > > > > > [4], [6] and [7].In summary, with the code in branch, you can > > > > create > > > > > > > tables > > > > > > > > with region replicas, do gets / multi gets and scans using > > > TIMELINE > > > > > > > > consistency with high availability. Region replicas > > periodically > > > > scan > > > > > > the > > > > > > > > files of the primary and pick up flushed / committed files. > The > > > RPC > > > > > > > paths / > > > > > > > > assignment, balancing etc are pretty stable. However some > more > > > > > > > performance > > > > > > > > analysis and tuning is needed. Phase 2 work is being worked > on > > > > under > > > > > > > > HBASE-11183, and we have some working prototype for > > > > async-replicating > > > > > > and > > > > > > > > region splits. However, we believe even without those > features, > > > > this > > > > > > work > > > > > > > > is useable (especially for read-only/bulk load tables) , and > > can > > > be > > > > > > > > released as an experimental feature in 1.0. > > > > > > > > > > > > > > > > Please indicate your choice: > > > > > > > > > > > > > > > > [ ] +1 on yes, merge branch hbase-10070 to trunk. > > > > > > > > [ ] 0 on don't care > > > > > > > > [ ] -1 don't merge, because ... > > > > > > > > > > > > > > > > I'll keep the vote running for 7 days, and close it Mon 9th > of > > > > June, > > > > > > PDT. > > > > > > > > > > > > > > > > Here is my official +1. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Enis > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070 > > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved > > > > > > > > [3] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved > > > > > > > > [4] > > > > https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html > > > > > > > > [5] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0 > > > > > > > > [6] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time > > > > > > > > [7] https://issues.apache.org/jira/browse/HBASE-10070 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best regards, > > > > > > > > > > > > > > - Andy > > > > > > > > > > > > > > Problems worthy of attack prove their worth by hitting back. - > > Piet > > > > > Hein > > > > > > > (via Tom White) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > // Jonathan Hsieh (shay) > > > > > > // HBase Tech Lead, Software Engineer, Cloudera > > > > > > // j...@cloudera.com // @jmhsieh > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > // Jonathan Hsieh (shay) > > > > // HBase Tech Lead, Software Engineer, Cloudera > > > > // j...@cloudera.com // @jmhsieh > > > > > > > > > > > > > > > -- > > // Jonathan Hsieh (shay) > > // HBase Tech Lead, Software Engineer, Cloudera > > // j...@cloudera.com // @jmhsieh > > > > > > -- > Thanks, > Michael Antonov >
