Re: Unable to Find HbaseContext

2015-10-21 Thread Ted Yu
Hbase-spark module currently is only in master branch. The feature can be back ported to branch-1 in the future. Cheers > On Oct 20, 2015, at 11:26 PM, Amit Hora wrote: > > I included cloud era's spark on hbase and after resolving dependencies it is > working fine > > I believe hbase-spar

Re: Slow region moves

2015-10-21 Thread ramkrishna vasudevan
I think the forceful clearing of the blocks from the bucket cache is hurting in this case. I think it is worth opening a JIRA for this and work on a fix. Regards Ram On Wed, Oct 21, 2015 at 12:00 AM, Randy Fox wrote: > Hi Vlad, > > I tried it on a table and on a RegionServer basis and it appea

Re: Slow region moves

2015-10-21 Thread ramkrishna vasudevan
Seems that the BucketAllocator#freeBlock() is synchronized and hence all the bulk close that it tries to do will be blocked in the synchronized block. May be something like the IdLock has to be tried here? Regards Ram On Wed, Oct 21, 2015 at 4:20 PM, ramkrishna vasudevan < ramkrishna.s.vasude...

Re: wal.FSHLog: Slow sync cost: 143 ms, current pipeline:

2015-10-21 Thread Jeesoo Shin
FSHLog: Slow sync cost:143 ms, current ... I have seen this messages at recent performance test. This showed up when hdfs had hard time catching up. (I did a lot of put request. ) check read/write byte of hdfs jmx to confirm. Maybe when you added new RS server, loadbalance kicked in and many regio

Re: Slow region moves

2015-10-21 Thread Vladimir Rodionov
I wonder why disabling cache eviction on close does not work in a case of a bucket cache? I checked the code and did not find anything suspicious. It has to work. On Wed, Oct 21, 2015 at 3:52 AM, ramkrishna vasudevan < ramkrishna.s.vasude...@gmail.com> wrote: > Seems that the BucketAllocator#fre

Re: Slow region moves

2015-10-21 Thread Randy Fox
Maybe I am looking in the wrong place but Hstore::close() has the evictOnClose parameter hard coded to true: // close each store file in parallel CompletionService completionService = new ExecutorCompletionService(storeFileCloserThreadPool); for (final StoreFile f : result) { completionServic

Re: Bucket cache causes region server status page to use 1GB ram

2015-10-21 Thread James Hartshorn
Just to follow up, we built the patch that was made for this issue and are testing it now. The problem looks to be totally fixed and we are very happy with the improved performance and usability. Thanks everyone! From: Ted Yu Sent: Wednesday, October 7

Start hbase with replication mode

2015-10-21 Thread beeshma r
Hi i just want to hbase as a replication mode.As per documentation zookeeper must not be managed by HBase so created below settings *zookeeper zoo.cfg(/home/beeshma/zookeeper-3.4.6/cfg)* tickTime=2000 dataDir=/home/beeshma/zookeeper clientPort=2181 initLimit=5 syncLimit=2 *hbase-site.xml* h

Re: Slow region moves

2015-10-21 Thread Vladimir Rodionov
You are right, Randy This is the bug. Will you open JIRA? -Vlad On Wed, Oct 21, 2015 at 9:35 AM, Randy Fox wrote: > Maybe I am looking in the wrong place but Hstore::close() has the > evictOnClose parameter hard coded to true: > > // close each store file in parallel > CompletionService comple

Re: Slow region moves

2015-10-21 Thread Randy Fox
https://issues.apache.org/jira/browse/HBASE-14663 -r On 10/21/15, 10:35 AM, "Vladimir Rodionov" wrote: >You are right, Randy > >This is the bug. Will you open JIRA? > >-Vlad > >On Wed, Oct 21, 2015 at 9:35 AM, Randy Fox wrote: > >> Maybe I am looking in the wrong place but Hstore::close() ha

Re: Slow region moves

2015-10-21 Thread Vladimir Rodionov
Randy, You can try patch I just submitted. It is for master but I verified it on 1.0 branch as well. -Vlad On Wed, Oct 21, 2015 at 11:40 AM, Randy Fox wrote: > https://issues.apache.org/jira/browse/HBASE-14663 > > -r > > > > On 10/21/15, 10:35 AM, "Vladimir Rodionov" wrote: > > >You are right