{quote}
If MR framework is not deployed in the cluster, hbase still functions
normally (post merge).
{quote}If MR is not strong dependency for Master/RS, it is OK for me. And if MR not deployed, Backup/Restore feature could not be used, right? 2016-09-23 10:49 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: > If MR framework is not deployed in the cluster, hbase still functions > normally (post merge). > > In terms of build time dependency, we have long been depending on > mapreduce. Take a look at ExportSnapshot. > > Cheers > > On Thu, Sep 22, 2016 at 7:42 PM, Heng Chen <heng.chen.1...@gmail.com> wrote: > >> In our production cluster, it is a common case we just have HDFS and >> HBase deployed. >> If our Master/RS depend on MR framework (especially some features we >> have not used at all), it introduced another cost for maintain. I >> don't think it is a good idea. >> >> 2016-09-23 10:28 GMT+08:00 张铎 <palomino...@gmail.com>: >> > To be specific, for example, our nice Backup/Restore feature, if we think >> > this is not a core feature of HBase, then we could make it depend on MR, >> > and start a standalone BackupManager instance that submits MR jobs to do >> > periodical maintenance job. And if we think this is a core feature that >> > everyone should use it, then we'd better implement it without MR >> > dependency, like DLS. >> > >> > Thanks. >> > >> > 2016-09-23 10:11 GMT+08:00 张铎 <palomino...@gmail.com>: >> > >> >> I‘m -1 on let master or rs launch MR jobs. It is OK that some of our >> >> features depend on MR but I think the bottom line is that we should >> launch >> >> the jobs from outside manually or by other services. >> >> >> >> 2016-09-23 9:47 GMT+08:00 Andrew Purtell <andrew.purt...@gmail.com>: >> >> >> >>> Ok, got it. Well "shelling out" is on the line I think, so a fair >> >>> question. >> >>> >> >>> Can this be driven by a utility derived from Tool like our other MR >> apps? >> >>> The issue is needing the AccessController to decide if allowed? But >> nothing >> >>> prevents the user from running the job manually/independently, right? >> >>> >> >>> > On Sep 22, 2016, at 3:44 PM, Matteo Bertozzi < >> theo.berto...@gmail.com> >> >>> wrote: >> >>> > >> >>> > just a remark. my query was not about tools using MR (everyone i >> think >> >>> is >> >>> > ok with those). >> >>> > the topic was about: "are we ok with running MR jobs from Master and >> RSs >> >>> > code?" since this will be the first time we do this >> >>> > >> >>> > Matteo >> >>> > >> >>> > >> >>> >> On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <d...@hortonworks.com> >> >>> wrote: >> >>> >> >> >>> >> Very much agree; for tools like ExportSnapshot / Backup / Restore, >> it's >> >>> >> fine to be dependent on MR. MR is the right framework for such. We >> >>> should >> >>> >> also do compactions using MR (just saying :) ) >> >>> >> ________________________________________ >> >>> >> From: Ted Yu <yuzhih...@gmail.com> >> >>> >> Sent: Thursday, September 22, 2016 2:00 PM >> >>> >> To: dev@hbase.apache.org >> >>> >> Subject: Re: [DISCUSSION] MR jobs started by Master or RS >> >>> >> >> >>> >> I agree - backup / restore is in the same category as import / >> export. >> >>> >> >> >>> >> On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell < >> >>> andrew.purt...@gmail.com> >> >>> >> wrote: >> >>> >> >> >>> >>> Backup is extra tooling around core in my opinion. Like import or >> >>> export. >> >>> >>> Or the optional MOB tool. It's fine. >> >>> >>> >> >>> >>>> On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi < >> mberto...@apache.org> >> >>> >>> wrote: >> >>> >>>> >> >>> >>>> What's the latest opinion around running MR jobs from hbase >> (Master >> >>> or >> >>> >>> RS)? >> >>> >>>> >> >>> >>>> I remember in the past that there was discussion about not having >> MR >> >>> >> has >> >>> >>>> direct dependency of hbase. >> >>> >>>> >> >>> >>>> I think some of discussion where around MOB that had a MR job to >> >>> >> compact, >> >>> >>>> that later was transformed in a non-MR job to be merged, I think >> we >> >>> >> had a >> >>> >>>> similar discussion for log split/replay. >> >>> >>>> >> >>> >>>> the latest is the new Backup feature (HBASE-7912), that runs a MR >> job >> >>> >>> from >> >>> >>>> the master to copy data or restore data. >> >>> >>>> (backup is also "not really core" as in.. if you don't use backup >> >>> >> you'll >> >>> >>>> not end up running MR jobs, but this was probably true for MOB as >> in >> >>> >> "if >> >>> >>>> you don't enable MOB you don't need MR") >> >>> >>>> >> >>> >>>> any thoughts? do we a rule that says "we don't want to have hbase >> run >> >>> >> MR >> >>> >>>> jobs, only tool started manually by the user can do that". or can >> we >> >>> >>> start >> >>> >>>> adding MR calls around without problems? >> >>> >>> >> >>> >> >> >>> >> >> >> >> >>
