Re: question on "drain region servers"

2016-04-25 Thread Ted Yu
Please take a look at: bin/draining_servers.rb On Mon, Apr 25, 2016 at 8:12 PM, WangYQ wrote: > in hbase, I find there is a "drain regionServer" feature > > > if a rs is added to drain regionServer in ZK, then regions will not be > move to on these regionServers > >

question on "drain region servers"

2016-04-25 Thread WangYQ
in hbase, I find there is a "drain regionServer" feature if a rs is added to drain regionServer in ZK, then regions will not be move to on these regionServers but, how can a rs be add to drain regionServer, we add it handly or rs will add itself automaticly

Re: Slow sync cost

2016-04-25 Thread Saad Mufti
Thanks, the meaning makes sense now but I still need to figure out why we keep seeing this. How would I know if this is just us overloading the capacity of our system with too much write load vs some configuration problem? Thanks. Saad On Mon, Apr 25, 2016 at 4:25 PM, Ted Yu

Re: Hbase shell script from java

2016-04-25 Thread Saad Mufti
I don't know why you would want to do this all through the hbase shell if your main driver is Java, unless you have a lot of existing complicated scripts you want to leverage. Why not just write Java code against the standard Java hbase client? Or if you need different parameters for each time

RE: Hbase shell script from java

2016-04-25 Thread Saurabh Malviya (samalviy)
Thanks, That will work for me. I am just curious how people are doing in industry, Suppose take a case where you have more than 100 tables and need to modify table script a lot for each deployment for performance or other reasons. -Saurabh -Original Message- From: Saad Mufti

Re: Slow sync cost

2016-04-25 Thread Ted Yu
w.r.t. the pipeline, please see this description: http://itm-vm.shidler.hawaii.edu/HDFS/ArchDocUseCases.html On Mon, Apr 25, 2016 at 12:18 PM, Saad Mufti wrote: > Hi, > > In our large HBase cluster based on CDH 5.5 in AWS, we're constantly seeing > the following messages

Re: current leaseholder is trying to recreate file error with ProcedureV2

2016-04-25 Thread Ted Yu
Can you pastebin more of the master log ? Which version of hadoop are you using ? Log snippet from namenode w.r.t. state-0073.log may also provide some more clue. Thanks On Mon, Apr 25, 2016 at 12:56 PM, donmai wrote: > Hi all, > > I'm getting a strange

current leaseholder is trying to recreate file error with ProcedureV2

2016-04-25 Thread donmai
Hi all, I'm getting a strange error during table creation / disable in HBase 1.1.2: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException): failed to create file /hbase/MasterProcWALs/state-0073.log for

Slow sync cost

2016-04-25 Thread Saad Mufti
Hi, In our large HBase cluster based on CDH 5.5 in AWS, we're constantly seeing the following messages in the region server logs: 2016-04-25 14:02:55,178 INFO org.apache.hadoop.hbase.regionserver.wal.FSHLog: Slow sync cost: 258 ms, current pipeline:

Re: Balancing reads and writes

2016-04-25 Thread Stack
Make sure you are on an HBase that has this fix in it too: HBASE-15213 (append goes the same code path as increment). St.Ack On Mon, Apr 25, 2016 at 1:38 AM, Kevin Bowling wrote: > Yeah here's the config I ended up with. Out of the box it had really > severe blocking

Re: New blog post: HDFS HSM and HBase by Jingcheng Du and Wei Zhou

2016-04-25 Thread Heng Chen
That's great! We are ready to use SSD to improve read performance now. 2016-04-23 8:25 GMT+08:00 Stack : > It is well worth the read. It goes deep so is a bit long and I had to cut > it up to do Apache Blog sized bits. Start reading here: >

Re: Balancing reads and writes

2016-04-25 Thread Kevin Bowling
Yeah here's the config I ended up with. Out of the box it had really severe blocking for write bursts, reads are much better with this and handlers turned up a bit: hbase.ipc.server.callqueue.read.ratio 0.4 hbase.ipc.server.callqueue.scan.ratio 0.5

Re: Can not connect local java client to a remote Hbase

2016-04-25 Thread SOUFIANI Mustapha | السفياني مصطفى
Hi, Thanks Sreeram for replying. But could you please tell me how this could be done ? is it in the hbase-site.xml ? Thanks in advance 2016-04-23 7:47 GMT+01:00 Sreeram : > Hi Soufiani, > > Can you try changing your configuration to have region server listen on >