BlockCache for large scans.

2014-04-09 Thread gortiz
I've been reading the book definitive guide and hbase in action a little. I found this question from Cloudera that I'm not sure after looking some benchmarks and documentations from HBase. Could someone explain me a little about? . I think that when you do a large scan you should disable the

Lighter Map/Reduce on HBase

2014-04-09 Thread Henning Blohm
We operate a solution that stores large amounts of data in HBASE that needs to be available for online access. For efficient scanning, there are three pieces of data encoded in row keys (in particular a time dimension) and for other reasons some columns hold JSON encoded data. Currently,

Re: 答复: 答复: HBase Replication - Addition alone

2014-04-09 Thread Manthosh Kumar T
Hi Demai, I thought making the client communicate with Central Server frequently may not be efficient. In case of replication, I can do that periodically in bulk. On 7 April 2014 22:00, Demai Ni nid...@gmail.com wrote: I looks to me that. You like to have the small clusters

CDH5 b2 - Hbase 0.96 - REST not returning data

2014-04-09 Thread Juraj jiv
Hello, i have one table in Hbase with 250GB of data and have problem while using Hbase REST scanner. What i do is: 1. calling http://server:20550/table/scanner with POST and Content-Type: text/xml Scanner endRow=value2 startRow=value1 batch=100/Scanner 2. then getting header location and

hbase region server reboot steps

2014-04-09 Thread Rural Hunter
Hi, I have one region server which needs to be rebooted for server maintenance. The server hosts both the hadoop and hbase slave(hadoop2-hbase 0.96). What is the recommended steps to reboot it without impacting hbase service?

Re: hbase region server reboot steps

2014-04-09 Thread Jean-Marc Spaggiari
Hi Rural, Decomission the node, stop the processes, and reboot. You can look at the scripts in bin/ to help you with that. Like bin/graceful_stop.sh. JM 2014-04-09 9:57 GMT-04:00 Rural Hunter ruralhun...@gmail.com: Hi, I have one region server which needs to be rebooted for server

Re: hbase region server reboot steps

2014-04-09 Thread Ted Yu
You can use bin/graceful_stop.sh to stop the region server process. # Move regions off a server then stop it. Optionally restart and reload. # Turn off the balancer before running this script. After that, you can stop hadoop (datanode, etc) On Wed, Apr 9, 2014 at 7:57 AM, Rural Hunter

Re: BlockCache for large scans.

2014-04-09 Thread Ted Yu
Please take a look at http://www.n10k.com/blog/blockcache-101/ For D, hbase.regionserver.global.memstore.size is specified in terms of percentage of heap. Unless you enable HBASE-5349 'Automagically tweak global memstore and block cache sizes based on workload' On Wed, Apr 9, 2014 at 12:24 AM,

Re: hbase region server reboot steps

2014-04-09 Thread Rural Hunter
Thanks. What if I'm not able to login the region server(both ssh and local)? I have to reboot and check the server because of this serious problem. 于 2014/4/9 22:01, Ted Yu 写道: You can use bin/graceful_stop.sh to stop the region server process. # Move regions off a server then stop it.

Re: hbase region server reboot steps

2014-04-09 Thread Rural Hunter
Actually I have to do a hard reboot. Let me provide more info about the problem: Except the ssh service(ssh error is: ssh_exchange_identification: Connection closed by remote host) and local login problem, other services are running fine on the server(including http/ftp/hbase/hadoop etc). 于

Re: hbase region server reboot steps

2014-04-09 Thread Jean-Marc Spaggiari
Hum. Disable load balancer, and move all the regions manually to other hosts using the shell? Then hard restart it? JM 2014-04-09 10:26 GMT-04:00 Rural Hunter ruralhun...@gmail.com: Actually I have to do a hard reboot. Let me provide more info about the problem: Except the ssh service(ssh

Re: hbase region server reboot steps

2014-04-09 Thread Ted Yu
Rural: Take a look at: http://hbase.apache.org/book.html#decommission especially 15.3.1.1 On Wed, Apr 9, 2014 at 8:28 AM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: Hum. Disable load balancer, and move all the regions manually to other hosts using the shell? Then hard restart it?

Re: BlockCache for large scans.

2014-04-09 Thread gortiz
Pretty interested the link, I'll keep it in my favorites. On 09/04/14 16:07, Ted Yu wrote: Please take a look at http://www.n10k.com/blog/blockcache-101/ For D, hbase.regionserver.global.memstore.size is specified in terms of percentage of heap. Unless you enable HBASE-5349 'Automagically

Re: 答复: 答复: HBase Replication - Addition alone

2014-04-09 Thread Demai Ni
Manthosh, not sure doing replication periodically is the right way to go. I assume you will manually start/stop replication from Master? anyway, if it fits your business needs, give it a shoot. Let's know how it goes. Cheers. Demai On Wed, Apr 9, 2014 at 6:10 AM, Manthosh Kumar T

ZooKeeper available but no active master location found

2014-04-09 Thread Margusja
Hi I downloaded and installed hortonworks sandbox 2.0 for virtualbox. HBase version is: 0.96.0.2.0.6.0-76-hadoop2, re6d7a56f72914d01e55c0478d74e5cfd3778f231 [hbase@sandbox twitbase-master]$ cat /etc/hosts # Do not remove the following line, or various programs # that require network

Re: CDH5 b2 - Hbase 0.96 - REST not returning data

2014-04-09 Thread stack
On Wednesday, April 9, 2014 6:18:35 AM UTC-7, Juraj jiv wrote: Hello, i have one table in Hbase with 250GB of data and have problem while using Hbase REST scanner. What i do is: 1. calling http://server:20550/table/scanner with POST and Content-Type: text/xml Scanner endRow=value2

HBase 0.98.1 split policy change?

2014-04-09 Thread Jean-Marc Spaggiari
Hi, I try to chance my HBase 0.98.1 split policy to be ConstantSizeRegionSplitPolicy so I have updated my hbase-site.xml do be this: property namehbase.regionserver.region.split.policy/name valueorg.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy/value /property Then on

HBase Unable to find Region Server - No Exception being thrown

2014-04-09 Thread kanwal
I'm currently running into an issue on my local setup where my application is unable to connect to the hbase table but I'm successfully able to query the table using hbase shell. I'm using HTable client to make the connection and would expect to get an error after certain retries when it's unable

Re: HBase 0.98.1 split policy change?

2014-04-09 Thread Jean-Marc Spaggiari
Ok. From the logs, I can clearly see that my settings are not getting use: 2014-04-09 15:35:03,737 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy: ShouldSplit because info size=2242479890, sizeToCheck=2147483648, regionsWithCommonTable=2 I will try to figure why

restore snapshot on another cluster

2014-04-09 Thread Artem Ervits
Hello all, When I take a snapshot on cluster 1, copy it to cluster 2 using ExportSnapshot utility, what permissions should I set on the snapshot to be able to clone it into a new table? I matched the permissions of the external snapshot to the permissions of any local snapshots on cluster 2

Re: BlockCache for large scans.

2014-04-09 Thread Asaf Mesika
The Jira says it's enabled by auto. Is there an official explaining this feature? On Wednesday, April 9, 2014, Ted Yu yuzhih...@gmail.com wrote: Please take a look at http://www.n10k.com/blog/blockcache-101/ For D, hbase.regionserver.global.memstore.size is specified in terms of percentage

Re: BlockCache for large scans.

2014-04-09 Thread Ted Yu
Didn't quite get what you mean, Asaf. If you're talking about HBASE-5349, please read release note of HBASE-5349. By default, memstore min/max range is initialized to memstore percent: globalMemStorePercentMinRange = conf.getFloat( MEMSTORE_SIZE_MIN_RANGE_KEY,

Re: restore snapshot on another cluster

2014-04-09 Thread Ted Yu
Can you give us some more detail such as: the HBase release you're using the stack trace of permission error I assume you have read 15.8.7 and 15.8.8 of: http://hbase.apache.org/book.html#ops.snapshots Cheers On Wed, Apr 9, 2014 at 3:08 PM, Artem Ervits are9...@nyp.org wrote: Hello all,

Re: ZooKeeper available but no active master location found

2014-04-09 Thread Ted Yu
Have you modified pom.xml of twitbase ? If not, this is the dependency you get: dependency groupIdorg.apache.hbase/groupId artifactIdhbase/artifactId version0.92.1/version 0.92.1 and 0.96.0 are not compatible. Cheers On Wed, Apr 9, 2014 at 10:58 AM, Margusja mar...@roo.ee

Re: restore snapshot on another cluster

2014-04-09 Thread Artem Ervits
I read it thanks. I'm getting privilege access exception hbase user on write permission. I don't have the exception handy. I will supply it shortly. I'm using 0.96.1.1 on hadoop 2. Artem Ervits Data Analyst New York Presbyterian Hospital - Original Message - From: Ted Yu

RE: restore snapshot on another cluster

2014-04-09 Thread Artem Ervits
2014-03-21 11:35:36,998 FATAL [master:server:6] master.HMaster: Unhandled exception. Starting shutdown. org.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode=/apps:hdfs:hdfs:drwxr-xr-x at

Re: restore snapshot on another cluster

2014-04-09 Thread Ted Yu
You can either give user hbase write access to /apps, or use another directory where user hbase can write to. Cheers On Wed, Apr 9, 2014 at 6:28 PM, Artem Ervits are9...@nyp.org wrote: 2014-03-21 11:35:36,998 FATAL [master:server:6] master.HMaster: Unhandled exception. Starting shutdown.

Re: HBase Unable to find Region Server - No Exception being thrown

2014-04-09 Thread Shengjun Xin
Does hbase-regionserver log have some error message? On Thu, Apr 10, 2014 at 3:27 AM, kanwal kanwal...@gmail.com wrote: I'm currently running into an issue on my local setup where my application is unable to connect to the hbase table but I'm successfully able to query the table using hbase

Re: HBase Unable to find Region Server - No Exception being thrown

2014-04-09 Thread kanwal
No. Couldn't find any error in the log. On Wed, Apr 9, 2014 at 10:26 PM, Shengjun Xin [via Apache HBase] ml-node+s679495n4058051...@n3.nabble.com wrote: Does hbase-regionserver log have some error message? On Thu, Apr 10, 2014 at 3:27 AM, kanwal [hidden