??????Query occasionally respond very slowly

The hbase version is 0.98.6-cdh5.2.0. The HDFS version is 2.5.0-cdh5.2.0. -- -- ??: "-";<175998...@qq.com>; : 2018??1??23??(??) 2:50 ??: "user"; : Query occasionally respond very slowly Recently, query occasionally r

Query occasionally respond very slowly

Recently, query occasionally respond very slowly.These queries usually return quickly, within a few milliseconds.But occasionally it gets very slow, reaching more than 20 seconds.I looked at the gc log and there was no full gc happening. A regionserver log is as follows?? 2018-01-22 16:38:13,58

Re: HbaseTestingUtlity with hbase-shaded-client and hbase-shaded-server

Any one any thoughts? Is there any workaround? On Mon, Jan 22, 2018 at 12:03 PM, Debraj Manna wrote: > Hi > > Is the above exception related to the below open issue > https://issues.apache.org/jira/browse/HBASE-15666 ? > > Thanks, > > On Sun, Jan 21, 2018 at 10:43 PM, Debraj Manna > wrote: > >>

Re: Coprocessor prePut

bq. can call context.complete() to skip other coprocessors. FYI in hbase-2, complete() has been folded into bypass(). See javadoc of bypass() for details. On Mon, Jan 22, 2018 at 9:34 PM, Yang Zhang wrote: > In fact, I prevent normal writes, and put the same data with a different > row key(Exce

Re: Coprocessor prePut

In fact, I prevent normal writes, and put the same data with a different row key(Except row key, they have the same columns and same data). I have already solved the infinite call by add and check some flag attribute to Put. If the flag exist then skip my code. This will prevent the infinite call

Re: Coprocessor prePut

Your prePut would write to a different column in the table, right ? Otherwise do you prevent normal writes from getting into the same column(s) ? If your prePut writes to dedicated column(s), you can check the presence of such column(s) so that the call is not infinite. Cheers On Mon, Jan 22, 20

Re: Coprocessor prePut

Yes, It is the same table. 2018-01-23 1:46 GMT+08:00 Ted Yu : > Can you clarify your use case ? > > bq. put a data into table > > Does your coprocessor write to the same table which receives user data ? > > Cheers > > On Mon, Jan 22, 2018 at 4:24 AM, Yang Zhang > wrote: > > > Hello Everyone > >

Re: How spark writes to HBASE

Setting hbase.bulkload.locality.sensitive.enabled to true and hbase.mapreduce.hfileoutputformat.table.name to the would do the magic to keep locality with best efforts during bulkload, FYI. More details please refer to HBASE-12596 Best Regards,

Re: ValueFilter finding old versions of cells

Hi, If i understand your question correctly, if your interested in getting the value FOO you should change your value filter to as below scan 't1', { COLUMNS => 'f1:a', FILTER => "ValueFilter( =, 'binaryprefix *:FOO'* )" } instead of binaryprefix:foo' If you query after before major compaction

ValueFilter finding old versions of cells

Hi, When doing a scan with a ValueFilter, I get an old cell value out, even with VERSIONS => 1 set for the table. hbase(main):003:0> create 't1', 'f1' 0 row(s) in 1.8020 seconds hbase(main):005:0> put 't1', 'foo', 'f1:a', 'foo' 0 row(s) in 0.1260 seconds hbase(main):006:0> put 't1', 'foo', 'f1:a'

RE: [ANNOUNCE] Apache Phoenix 4.13.2 for CDH 5.11.2 released

This release is an effort of the Apache Phoenix community, not Cloudera. That's why they don't know (yet) about it. It's quite uncommon for an Apache project to support a vendor platform, but it was discussed and agreed in the development mailing list. I think that the PCM is also open to embrace

RE: hbase backups

Master slave architecture is fine but why you want to take the snapshot on slave cluster. Your replication should covers disaster recovery scenario. Regards, CM +1 201 763 1656 -Original Message- From: dkazhdan [mailto:dima.kazh...@td.com] Sent: Monday, January 22, 2018 12:56 AM To: u

RE: [ANNOUNCE] Apache Phoenix 4.13.2 for CDH 5.11.2 released

Is there any documentation on cloudera site where it is announced about this news. I checked with cloudera team and they are still not aware of this. Regards, CM +1 201 763 1656 -Original Message- From: James Taylor [mailto:jamestay...@apache.org] Sent: Saturday, January 20, 2018 6:02

Re: Coprocessor prePut

Can you clarify your use case ? bq. put a data into table Does your coprocessor write to the same table which receives user data ? Cheers On Mon, Jan 22, 2018 at 4:24 AM, Yang Zhang wrote: > Hello Everyone > > I am using the coprocessor and want to put another data when > someone put

Re: How spark writes to HBASE

I did a search in SHC for saveAsNewHadoop (case insensitive) - there was no match. I suggest you use SHC forum for related questions. On Mon, Jan 22, 2018 at 9:07 AM, vignesh wrote: > It would be similar to case 2 right. Say for example in spark I read a file > of size 512mb which would span 4

Re: How spark writes to HBASE

It would be similar to case 2 right. Say for example in spark I read a file of size 512mb which would span 4 cores(if block size is 128). Executor will be spanned based on data locality and if the executor is launched machine1,2,3,4. May be if block3 region is handled by machine6, then when I bulk

Re: How spark writes to HBASE

For case 1, HFile would be loaded into the region (via staging directory). Please see: http://hbase.apache.org/book.html#arch.bulk.load On Mon, Jan 22, 2018 at 8:52 AM, vignesh wrote: > If it is a bulk load I use spark hbase connector provided by hortonworks. > For time series writes I use norm

RE: How spark writes to HBASE

So in cases like spark the write to memstore will be via network if the executor runs on different machine than the region server machine which is responsible to take the puts? On Jan 22, 2018 22:24, "Dave Birdsall" wrote: > There are some engines that will do this. Apache Trafodion for example

RE: How spark writes to HBASE

There are some engines that will do this. Apache Trafodion for example will hash partition results to be inserted into a table in HBase so that the puts are done locally. -Original Message- From: Ted Yu [mailto:yuzhih...@gmail.com] Sent: Monday, January 22, 2018 8:47 AM To: user@hbase.a

Re: How spark writes to HBASE

If it is a bulk load I use spark hbase connector provided by hortonworks. For time series writes I use normal hbase client API's. So does that mean in case 2(client API write) the write to memstore will happen via network? In case 1(bulk load)the HFile will be moved to the region which is suppose

Re: How spark writes to HBASE

Which connector do you use to perform the write ? bq. Or spark will wisely launch an executor on that machine I don't think that is the case. Multiple writes may be performed which would end up on different region servers. Spark won't provide the affinity described above. On Mon, Jan 22, 2018 at

How spark writes to HBASE

Hi, I have a Spark job which reads some timeseries data and pushes that to HBASE using HBASE client API. I am executing this Spark job on a 10 node cluster. Say at first when spark kicks off it picks machine1,machine2,machine3 as its executors. Now when the job inserts a row to HBASE. Below is wha

Coprocessor prePut

Hello Everyone I am using the coprocessor and want to put another data when someone put a data into table. So I try prePut(), but when you call HRegion.put(), this will call my coprocessor's prePut function again, and it will cause an dead loop. My code looks like below, If anyone