There is just one log file with "-MASTER-" in its name, the following is
the result of the command "*ls -R -l /var/log/hbase*"
[root@ec1 spark]# ls -R -l /var/log/hbase
/var/log/hbase:
total 136840
drwx------ 2 hbase hbase 4096 Jul 21 23:37 audit
-rw-r--r-- 1 hbase hbase 125902996 Oct 6 02:20
hbase-cmf-hbase-CANARY-ec1.hadoop.com.log.out
-rw-r--r-- 1 hbase hbase 1424163 Oct 5 09:38
hbase-cmf-hbase-MASTER-ec1.hadoop.com.log.out
-rw-r--r-- 1 hbase hbase 12774867 Oct 6 01:39
hbase-cmf-hbase-REGIONSERVER-ec1.hadoop.com.log.out
/var/log/hbase/audit:
total 0
So obviously there is no other log file with "-MASTER-" in its name.
The following is the content of the log4j settings file "
*/etc/hbase/conf.cloudera.hbase/log4j.properties*"
log4j.rootLogger=${hbase.root.logger}
hbase.root.logger=INFO,console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{2}: %m%n
2014-10-06 10:16 GMT+08:00 Sean Busbey <[email protected]>:
> Can you put your log4j settings for master in a pastebin and send us the
> link?
>
> --
> Sean
> On Oct 5, 2014 8:22 PM, "Tao Xiao" <[email protected]> wrote:
>
> > Hi Ted,
> >
> > Actually there is no more contents in the master log --- what you
> > saw is the snippet of master log from the time I executed "*balancer*" to
> > the end of the master log.
> >
> > I excerpted the snippet from the master log file
> > "/var/log/hbase/hbase-cmf-hbase-MASTER-ec1.hadoop.com.log.out" in the
> HBase
> > master node, I suppose that is the right master log file.
> >
> > Well, there is no more log info appended to that master log file from
> > yesterday till now, does it suggest something exceptional ?
> >
> > 2014-10-05 22:24 GMT+08:00 Ted Yu <[email protected]>:
> >
> > > bq. Client=root//10.196.111.116 set balanceSwitch=true
> > > The above was the only log pertaining to balancer. There was no region
> in
> > > trnasition.
> > >
> > > Can you give larger snippet of master log ?
> > >
> > > Cheers
> > >
> > > On Sun, Oct 5, 2014 at 2:28 AM, Tao Xiao <[email protected]>
> > wrote:
> > >
> > > > Hi all, I checked what all you said.
> > > >
> > > >
> > > > #1. Balancer
> > > > I used the command "*balance_switch true*" in the HBase shell and
> that
> > > > command returned true, indicating that the load balancer was running.
> > > Then
> > > > I used the command "*balancer*" to prompt HBase to load balance its
> > > > regions. However after more than 5 minutes, region assignment for the
> > > table
> > > > I mentioned in the first email did not change at all, see this screen
> > > shot
> > > > <http://imgbin.org/index.php?page=image&id=19618>. I checked the
> > master
> > > > log since the time I manaully trigger a balance operation by the
> > command
> > > "
> > > > *balancer*", and here is the master log <
> http://pastebin.com/PagJeB1f
> > >.
> > > > Since the load balancer is running all the time, it's weird to see
> that
> > > > regions of the table in concern were never balanced.
> > > >
> > > >
> > > > #2. Balancer configuration
> > > > I'm using CDH 5.1.0 and I did not find where to set balance
> > configuration
> > > > in Cloudera Manager. Could anybody tell me how to ?
> > > >
> > > >
> > > > #3. HBASE-12139 StochasticLoadBalancer
> > > > My HBase cluster has about ten tables, and none of these tables has
> > over
> > > 20
> > > > regions. The HBase cluster consisted of 5 nodes so the issue
> > HBASE-12139
> > > > does not apply to my problem.
> > > >
> > > >
> > > >
> > > > Thanks
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > 2014-10-03 20:31 GMT+08:00 Jean-Marc Spaggiari <
> > [email protected]
> > > >:
> > > >
> > > > > It depends on the time between the 2 screen-shots. But if there is
> > > more
> > > > > than 5 minutes (default config), then you are correct and it should
> > > have
> > > > > been a minimum balanced.
> > > > >
> > > > > 2014-10-03 3:30 GMT-04:00 Qiang Tian <[email protected]>:
> > > > >
> > > > > > according to the 2 pictures, looks the balancer not run?
> > > > > >
> > > > > > On Fri, Oct 3, 2014 at 1:45 AM, Ted Yu <[email protected]>
> > wrote:
> > > > > >
> > > > > > > See also
> > > > > > > HBASE-12139 StochasticLoadBalancer doesn't work on large
> lightly
> > > > loaded
> > > > > > > clusters
> > > > > > >
> > > > > > > Cheers
> > > > > > >
> > > > > > > On Thu, Oct 2, 2014 at 10:35 AM, Jean-Marc Spaggiari <
> > > > > > > [email protected]> wrote:
> > > > > > >
> > > > > > > > Also, What is the balancer configured on your cluster? Which
> > > HBase
> > > > > > > version?
> > > > > > > > If you have a 200 region table in a cluster with another
> pretty
> > > > small
> > > > > > 13
> > > > > > > > region table like the one you have, chances are high for this
> > 2nd
> > > > > table
> > > > > > > to
> > > > > > > > not be perfectly balanced over the cluster compared to the
> > first
> > > > one.
> > > > > > > >
> > > > > > > > JM
> > > > > > > >
> > > > > > > > 2014-10-02 13:30 GMT-04:00 Nick Dimiduk <[email protected]
> >:
> > > > > > > >
> > > > > > > > > Is the balancer running? If it's running but unable to act,
> > you
> > > > > > should
> > > > > > > > see
> > > > > > > > > some explanation in the Master logs.
> > > > > > > > >
> > > > > > > > > On Wed, Oct 1, 2014 at 8:27 PM, Tao Xiao <
> > > > [email protected]
> > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > I have a HBase table containing 40 million records.
> > Checking
> > > > > HBase
> > > > > > > UI,
> > > > > > > > I
> > > > > > > > > > could see that this table was not spread evenly across
> the
> > > > > cluster.
> > > > > > > Two
> > > > > > > > > > nodes, *ec1.hadoop.com:60020 <
> http://ec1.hadoop.com:60020
> > >*
> > > > and
> > > > > > > > > > *ec5.hadoop.com:60020
> > > > > > > > > > <http://ec5.hadoop.com:60020>*, for example, has only
> one
> > > > region
> > > > > > > each,
> > > > > > > > > > while other nodes have more regions. Here is the screen
> > shot
> > > > > > > > > > <http://imgbin.org/index.php?page=image&id=19571>.
> > > > > > > > > >
> > > > > > > > > > I tried to split that table by clicking the button
> "Split".
> > > > > Later,
> > > > > > > each
> > > > > > > > > > region of that table was split into two regions, but all
> > > > regions
> > > > > > for
> > > > > > > > that
> > > > > > > > > > table as a whole were still unevenly spread across the
> > > cluster.
> > > > > > Here
> > > > > > > > > > is a screen
> > > > > > > > > > shot after I click the button "Split"
> > > > > > > > > > <http://imgbin.org/index.php?page=image&id=19572>.
> > > > > > > > > >
> > > > > > > > > > So I'd like to know how to adjust the regions for a table
> > so
> > > > that
> > > > > > all
> > > > > > > > > > regions for that table can be evenly spread across the
> > > cluster.
> > > > > > > > > >
> > > > > > > > > > Thanks
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
