>> and >> backups too, but don't want to bother having to install and configure YARN >> just for that, as well as removing resources from HBase to give it to
Any suggestions on how to do bulk data move with transformation from/to HBase cluster w/o MapReduce? Opposition to M/R does not make sense imo, as since we have a lot of tools in HBase which depend on MapReduce: CountRows CountCells Import Export ImportTsv ExportTsv CopyTable VerifyReplication ExportSnapshot and new backup create/restore of course. -Vlad On Thu, Sep 22, 2016 at 4:15 PM, Jean-Marc Spaggiari < jean-m...@spaggiari.org> wrote: > My 2¢: I have a strong preference for NOT having a dependency on MR > anywhere :( I run my HBase cluste without YARN. Just HBase and HDFS. I like > all the features that we built. Would love to be able to use MOBs and > backups too, but don't want to bother having to install and configure YARN > just for that, as well as removing resources from HBase to give it to > yarn.... > > JMS > > 2016-09-22 18:44 GMT-04:00 Matteo Bertozzi <theo.berto...@gmail.com>: > > > just a remark. my query was not about tools using MR (everyone i think is > > ok with those). > > the topic was about: "are we ok with running MR jobs from Master and RSs > > code?" since this will be the first time we do this > > > > Matteo > > > > > > On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <d...@hortonworks.com> > wrote: > > > > > Very much agree; for tools like ExportSnapshot / Backup / Restore, it's > > > fine to be dependent on MR. MR is the right framework for such. We > should > > > also do compactions using MR (just saying :) ) > > > ________________________________________ > > > From: Ted Yu <yuzhih...@gmail.com> > > > Sent: Thursday, September 22, 2016 2:00 PM > > > To: dev@hbase.apache.org > > > Subject: Re: [DISCUSSION] MR jobs started by Master or RS > > > > > > I agree - backup / restore is in the same category as import / export. > > > > > > On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell < > > andrew.purt...@gmail.com> > > > wrote: > > > > > > > Backup is extra tooling around core in my opinion. Like import or > > export. > > > > Or the optional MOB tool. It's fine. > > > > > > > > > On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <mberto...@apache.org > > > > > > wrote: > > > > > > > > > > What's the latest opinion around running MR jobs from hbase (Master > > or > > > > RS)? > > > > > > > > > > I remember in the past that there was discussion about not having > MR > > > has > > > > > direct dependency of hbase. > > > > > > > > > > I think some of discussion where around MOB that had a MR job to > > > compact, > > > > > that later was transformed in a non-MR job to be merged, I think we > > > had a > > > > > similar discussion for log split/replay. > > > > > > > > > > the latest is the new Backup feature (HBASE-7912), that runs a MR > job > > > > from > > > > > the master to copy data or restore data. > > > > > (backup is also "not really core" as in.. if you don't use backup > > > you'll > > > > > not end up running MR jobs, but this was probably true for MOB as > in > > > "if > > > > > you don't enable MOB you don't need MR") > > > > > > > > > > any thoughts? do we a rule that says "we don't want to have hbase > run > > > MR > > > > > jobs, only tool started manually by the user can do that". or can > we > > > > start > > > > > adding MR calls around without problems? > > > > > > > > > >
