Re: is it a good idea to disable tables not currently hot?

2016-03-18 13:14 GMT-04:00 Frank Luo : > >Compactions should not hold the writes except if there is to many stores > files. > > I believe too many store files (9 I think?) is indeed the trigger for > compact. I always do MajorCompact before inserting. > This is not required.

Re: why Hbase only split regions in one RegionServer

Balancer is not moving regions that are compacting, right? He is just pusing to much load on a non splitted table that will keep splitting and compacting like crazy until balancer get a chance to get in action. Pre-split / Balance. Problem solved. Jack, when the ingestion of data is done, is

Re: How to implement increment in an idempotent manner

What's about the other option where each bolt increment it's own column and at theend ou aggregate those few columns together? 2016-03-18 6:14 GMT-04:00 Sreeram : > The incremented field is more like an amount field that will be storing the > aggregate amount. Since the

Re: why Hbase only split regions in one RegionServer

What if someone doesn't know the distribution of their row keys? HBase should be able to handle this case. On Wed, Mar 16, 2016 at 7:18 AM, Jean-Marc Spaggiari < jean-m...@spaggiari.org> wrote: > Balancer is not moving regions that are compacting, right? He is just > pusing to much load on a

HBase as a Multi-Model Data Store

Hi, If anyone is trying to store JSON in HBase, you may be interested in: https://rayokota.wordpress.com/2016/03/17/hbase-as-a-multi-model-data-store/

Java Thrift Question: how to filter Integer values?

My program talks to a remote HBase via Thrift (v1). I want to scan a table with "SingleColumnValueFilter". http://hbase.apache.org/0.94/book/thrift.html *Question: * When creating a TScan object, what's the correct syntax to use for "filterString" if I want to match an Integer column value?

Re: why Hbase only split regions in one RegionServer

You're definitely better off today if you know what your data looks like and are able to set up your table ahead of time accordingly. But I think this is an area where HBase can and should do better. We should try to lower the bar and make it simpler for people to get started. HBase would

Re: Hbase in memory rest running in jenkins

No Jenkins is running on Unix box then what need to be done On Mar 16, 2016 8:12 PM, "Ted Yu" wrote: > bq. Do I need to configure winutils.exe > > Assuming the Jenkins runs on Windows, winutils.exe would be needed. > > Cheers > > On Wed, Mar 16, 2016 at 5:19 AM, Gaurav

Re: is it a good idea to disable tables not currently hot?

By default memsotre is 40%. Here it's 24%. There is a lot you might want to look at on your cluster and usecase :( 1) You might have long pause GCs causing issues. Think about offheap cache and reduce heap to less than 20GB 2) Way to many regions. Think about your usecases and tables design to

Re: Upgrade from 1.1.3 to 1.2.0 failed

Hello, did you find any anomalies? Michal On Thu, Mar 10, 2016 at 2:50 PM, Michal Medvecky wrote: > Hello, > > the log you pasted is from later time when I was playing with all kinds of > knobs to make things work. > > Here are logs from freshly deployed cluster: > >

Re: is it a good idea to disable tables not currently hot?

@Frank, regarding write amplification: 1. What is your flush size? default is 128 MB. You should increase your "hbase.hregion.memstore.flush.size" so that you dont run over the limit of store files. 2. Have a look at "hbase.regionserver.global.memstore.lowerLimit". 3. Your heap size is also too

Re: How to implement increment in an idempotent manner

All my Hbase processing is going to be inside any given bolt. So I cannot use the second option. On Fri, Mar 18, 2016 at 3:46 PM, Jean-Marc Spaggiari < jean-m...@spaggiari.org> wrote: > What's about the other option where each bolt increment it's own column and > at theend ou aggregate those few

Re: Upgrade from 1.1.3 to 1.2.0 failed

unscribe 2016-03-17 17:07 GMT+08:00 Michal Medvecky : > Hello, > > did you find any anomalies? > > Michal > > On Thu, Mar 10, 2016 at 2:50 PM, Michal Medvecky wrote: > > > Hello, > > > > the log you pasted is from later time when I was playing with all kinds

RE: is it a good idea to disable tables not currently hot?

0.98 on hdp 2.2 currently. Soon will be on hdp2.3.4, which has HBase 1.1.2. -Original Message- From: Jean-Marc Spaggiari [mailto:jean-m...@spaggiari.org] Sent: Friday, March 18, 2016 11:29 AM To: user Subject: Re: is it a good idea to disable tables not currently hot? Hi Frank, It

Re: How to increase Thrift maximum concurrent connection number?

I think what observed (16 concurrent connections) was due to the following parameter: hbase.thrift.minWorkerThreads whose default value is 16. On Fri, Mar 18, 2016 at 11:06 PM, Daniel wrote: > Hi, my Thrift API server seems to allow at most 16 concurrent connections. > Is

Kerberos auth using java exception

Hi, I am trying to write java code snippet to authenticate hbase using Kerberos, But I am getting attached exception. I tried all kind of config through code but no luck. -Saurabh ---Java Code HbaseGetPropertyValues properties = new HbaseGetPropertyValues();

Re: Kerberos auth using java exception

>From the code, looks like you were trying to authenticate as user test. But from the attached log: bq. getLoginUser :: root (auth:SIMPLE) FYI On Thu, Mar 17, 2016 at 8:33 PM, Saurabh Malviya (samalviy) < samal...@cisco.com> wrote: > > > Hi, > > > > I am trying to write java code snippet to

Re: Thrift Server: How to increase "max worker threads" and "max queued requests"

Maybe you could add more information about hbase version and which thrift(thrift1 or thrift2) ? 在 16/3/17 下午4:35， "Ted Yu" 写入: >See the following from hbase-default.xml > > >hbase.thrift.minWorkerThreads >16 >... > >hbase.thrift.maxWorkerThreads >1000

Re: why Hbase only split regions in one RegionServer

It's usually better to do a bit more work upstream to know about you table schema and keys instead of trying to fight that later. HBase will handle the case. As soon as the compactions will be done. It's just that it's not a recommanded way to proceed. Better to prevent instead of trying to fix

Re: is it a good idea to disable tables not currently hot?

Frank: Can you take a look at the following to see if it may help with your use case(s) ? HBASE-15181 A simple implementation of date based tiered compaction Cheers On Fri, Mar 18, 2016 at 9:58 AM, Frank Luo wrote: > There are two reasons I am hesitating going that route.

Re: Hbase in memory rest running in jenkins

bq. Do I need to configure winutils.exe Assuming the Jenkins runs on Windows, winutils.exe would be needed. Cheers On Wed, Mar 16, 2016 at 5:19 AM, Gaurav Agarwal wrote: > Hello > > I used hbasetestingutility to run test cases on Windows and configure > winutils.exe

How to implement increment in an idempotent manner

Hi, I am looking for suggestions from community on implementing HBase increment in a idempotent manner. My use case is a storm Hbase bolt that atomically increments a HBase counter. Replay of the storm bolt results in a double increment. Any suggestion on the approach to be taken is welcome.

Re: Hbase in memory rest running in jenkins

Thanks, Will confirm On Wed, Mar 16, 2016 at 9:24 PM, Ted Yu wrote: > maven, git (which I think you may have setup already) > > winutils.exe is out of the picture. > > On Wed, Mar 16, 2016 at 7:56 AM, Gaurav Agarwal > wrote: > > > No Jenkins is

Re: How to implement increment in an idempotent manner

At the beginning of your Storm bolt process can you not do a put of "0"? So it start back from scratch? Or else you will need to query the value, and keep the value to put it back if you need to replay your bolt Other option is, you increment a specific difference column and at the end if you are

Thrift Server: How to increase "max worker threads" and "max queued requests"

Hi, I find that the Thrift server will stop responding (the request hangs until timeout) when the number of concurrent requests reaches several hundred. I guess the problem is related to "max worker threads" and "max queued requests", according to the following console output on Thrift start:

Hbase in memory rest running in jenkins

Hello I used hbasetestingutility to run test cases on Windows and configure winutils.exe there in the class path. But when I have to run same test case on Jenkins where no hadoop is installed .Do I need to configure winutils.exe there or what needs to be done please share the info Thanks

is it a good idea to disable tables not currently hot?

We have a multi tenants environment and each client occupies x number of hbase regions. We currently have about 500 regions per region server and I understand the guideline is less than 200. So we need to reduce the region counts. Increasing region file size is no more an option because we are

Re: is it a good idea to disable tables not currently hot?

bq. By default memsotre is 40%. Here it's 24% bq. Memstore.lowerLimit=0.24 J-M: Looks like you misread the config Frank listed. On Fri, Mar 18, 2016 at 12:36 PM, Jean-Marc Spaggiari < jean-m...@spaggiari.org> wrote: > By default memsotre is 40%. Here it's 24%. There is a lot you might want to

java.lang.VerifyError: class com.google.protobuf.HBaseZeroCopyByteString overrides final method equals.(Ljava/lang/Object;)Z

i am using HBase 1.0.0-cdh5.5.1 i am hitting this exception when trying to write to Hbase following is the stack trace Exception in thread "main" java.lang.VerifyError: class com.google.protobuf.HBaseZeroCopyByteString overrides final method equals.(Ljava/lang/Object;)Z at

How to increase Thrift maximum concurrent connection number?

Hi, my Thrift API server seems to allow at most 16 concurrent connections. Is there a way to raise this limit? I cannot find a clue in the documentation. Thanks a lot. Daniel