Jingcheng: Let us know if you need anything for composition of the vote thread.
On Tue, Jul 14, 2015 at 3:58 PM, Andrew Purtell <apurt...@apache.org> wrote: > That's great, and I also appreciate the time spent discussing on this > thread. I plan to vote yes. > > On Tue, Jul 14, 2015 at 3:57 PM, Jonathan Hsieh <j...@cloudera.com> wrote: > > > Since Jingcheng did the majority of the work, I'd like him to stat the > > thread vote thread. It should happen in the next few days. > > > > Jon > > > > On Tue, Jul 14, 2015 at 3:53 PM, Andrew Purtell <apurt...@apache.org> > > wrote: > > > > > Start a vote thread? This says DISCUSSION > > > > > > On Tue, Jul 14, 2015 at 2:05 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > > > There have been several iterations since the first mega patch was > > posted. > > > > > > > > Currently QA run is green. > > > > > > > > IntegrationTestIngestWithMOB has been run which passes. > > > > > > > > I want to give +1 for merging to master branch. > > > > > > > > On Tue, Jul 7, 2015 at 9:37 PM, Anoop John <anoop.hb...@gmail.com> > > > wrote: > > > > > > > > > +1 > > > > > > > > > > We will work on making it a Vote thread. > > > > > > > > > > -Anoop- > > > > > > > > > > On Wed, Jul 8, 2015 at 9:46 AM, ramkrishna vasudevan < > > > > > ramkrishna.s.vasude...@gmail.com> wrote: > > > > > > > > > > > +1 to start the voting again. > > > > > > > > > > > > On Wed, Jul 8, 2015 at 9:07 AM, Ted Yu <yuzhih...@gmail.com> > > wrote: > > > > > > > > > > > > > Looks like all MOB-related JIRAs have been resolved. > > > > > > > > > > > > > > Should the voting process be resumed ? > > > > > > > > > > > > > > Cheers > > > > > > > > > > > > > > On Thu, May 28, 2015 at 5:48 PM, Anoop John < > > anoop.hb...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Yes Andy. The sweep tool is completely optional now. We have > a > > > > chore > > > > > > > doing > > > > > > > > the compaction, like we trigger auto major compaction. We > can > > > > > > configure > > > > > > > > the interval. Auto can be turned off and user can explicitly > > call > > > > > also. > > > > > > > We > > > > > > > > have shell and API support. > > > > > > > > > > > > > > > > Anoop > > > > > > > > > > > > > > > > On Friday, May 29, 2015, Andrew Purtell <apurt...@apache.org > > > > > > wrote: > > > > > > > > > MOB references in cells won't find their value if the MOB > > hfile > > > > has > > > > > > > been > > > > > > > > > corrupted. Dealing with that would be like any other > > corrupted > > > > > HFile, > > > > > > > > > understood. The dangling references make thinking about > > > (partial) > > > > > > > > recovery > > > > > > > > > and repair interesting. > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thursday, May 28, 2015, Jingcheng Du < > > > jingcheng...@intel.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > >> Andrew Purtell wrote > > > > > > > > >> > HBCK can check and sideline dangling reference files. I > > > think > > > > of > > > > > > MOB > > > > > > > > >> files > > > > > > > > >> > as "core enough" auxiliary files that need some > support. I > > > > > suppose > > > > > > > > unlike > > > > > > > > >> > reference files their presence or absence won't produce > a > > > > region > > > > > > > open > > > > > > > > >> > failure, we would see dangling pointers later when tying > > to > > > > > > service > > > > > > > > >> > queries. (Yes?) Will that abort the RS? Pardon the > > ignorant > > > > > > > question, > > > > > > > > >> > normally I could check the code but I'm at the airport > on > > a > > > > > phone. > > > > > > > > >> > > > > > > > > >> Thanks for comments! > > > > > > > > >> In mob, usually the reference cells are committed after > the > > > mob > > > > > > files > > > > > > > > are > > > > > > > > >> done. I think it hardly happens that a reference cell > cannot > > > > find > > > > > > its > > > > > > > > mob > > > > > > > > >> file. > > > > > > > > >> Even if there's a dangling reference cell, the RS won't be > > > > > aborted, > > > > > > a > > > > > > > > empty > > > > > > > > >> cell is returned instead. > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> Andrew Purtell wrote > > > > > > > > >> > On that subject, I should file follow up issues for more > > > check > > > > > and > > > > > > > > repair > > > > > > > > >> > options for HFiles. We should be able to detect missing > or > > > > > corrupt > > > > > > > > files > > > > > > > > >> > of > > > > > > > > >> > all variety: HFile, reference, MOB. This may require an > > > > > expensive > > > > > > > scan > > > > > > > > >> > over > > > > > > > > >> > lots of files, but this is like fsck full disk surface > > scans > > > > and > > > > > > > those > > > > > > > > >> > have > > > > > > > > >> > similar costs. Providing MR based tools is fine but we > > > should > > > > > have > > > > > > > > >> > multithreaded tools that can stand in if a MR runtime is > > not > > > > > > > > available. > > > > > > > > >> > Import, Export, VerifyReplication...all of these tools > are > > > in > > > > a > > > > > > > > >> different, > > > > > > > > >> > lesser, class than integrity and repair tools, in my > > > opinion. > > > > > > Since > > > > > > > > MOB > > > > > > > > >> > will likely be merged into trunk by then I'll be sure to > > > > include > > > > > > > it. I > > > > > > > > >> > agree it's not fair to ask more of MOB then what we have > > now > > > > for > > > > > > > > HFile. > > > > > > > > >> > > > > > > > > >> To detect the corrupt files, some code are needed in the > > file > > > > > > checker > > > > > > > to > > > > > > > > >> check mob files after knowing it's a mob-enabled column. > > > > > > > > >> To detect the missing or dangling reference cells, I think > > we > > > > have > > > > > > to > > > > > > > do > > > > > > > > a > > > > > > > > >> full-table scan, like what is done now in HFile.main. > > > > > > > > >> We can do that. > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> -- > > > > > > > > >> View this message in context: > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-hbase.679495.n3.nabble.com/DISCUSSION-Merge-of-the-hbase-11339-mob-branch-into-master-tp4071644p4071911.html > > > > > > > > >> Sent from the HBase Developer mailing list archive at > > > > Nabble.com. > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Best regards, > > > > > > > > > > > > > > > > > > - Andy > > > > > > > > > > > > > > > > > > Problems worthy of attack prove their worth by hitting > back. > > - > > > > Piet > > > > > > > Hein > > > > > > > > > (via Tom White) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > > > > - Andy > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein > > > (via Tom White) > > > > > > > > > > > -- > > // Jonathan Hsieh (shay) > > // HBase Tech Lead, Software Engineer, Cloudera > > // j...@cloudera.com // @jmhsieh > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >
