RE: Use experience and performance data of offheap from Alibaba online cluster

2016-11-21 Thread Du, Jingcheng
Thanks Yu. You disabled the MSLAB pool in the running, why did you choose to disable it? I compared the performance with and without MSLAB pool once before, the GC pause can be improved if the MSLAB pool was enabled (I used G1GC at that time). Maybe the pool can help in your case? Regards, Jin

RE: Use experience and performance data of offheap from Alibaba online cluster

2016-11-18 Thread Du, Jingcheng
Thanks Yu for the sharing, great achievements. It seems the images cannot be displayed? Maybe just me? Regards, Jingcheng From: Yu Li [mailto:car...@gmail.com] Sent: Friday, November 18, 2016 4:11 PM To: u...@hbase.apache.org; dev@hbase.apache.org Subject: Use experience and performance data of o

RE: Data loss in MOB snapshot and clone?

2016-10-14 Thread Du, Jingcheng
t time to do some testing next week on this and see if I can come up with a reproducible test. I can confirm for non-MOB is it all fine, and fields below the MOB threshold were not lost in the original process. Cheers, Tim On Thu, Oct 13, 2016 at 5:31 PM, Du, Jingcheng wrote: > Hi Tim, &

RE: Data loss in MOB snapshot and clone?

2016-10-13 Thread Du, Jingcheng
thus broken references and data loss. Or am I misunderstanding you please? On Thu, Oct 13, 2016 at 9:45 AM, Du, Jingcheng wrote: > Hi Tim, > > > was this running a background task to copy the MOB data when the > snapshot was cloned and I just deleted the source before the copy wa

RE: Data loss in MOB snapshot and clone?

2016-10-13 Thread Du, Jingcheng
Hi Tim, > was this running a background task to copy the MOB data when the snapshot was > cloned and I just deleted the source before the copy was complete? The MOB data can be copied when mob compaction happens. But the MOB files should not be deleted even if they are not copied and after the s

RE: Should we remove Mob Sweeper MR job

2016-10-10 Thread Du, Jingcheng
It would be fine to remove it. Regards, Jingcheng -Original Message- From: Ted Yu [mailto:yuzhih...@gmail.com] Sent: Tuesday, October 11, 2016 6:03 AM To: dev@hbase.apache.org Subject: Re: Should we remove Mob Sweeper MR job Seems good cleanup to do. On Mon, Oct 10, 2016 at 2:55 PM, Ap

RE: RegionServer shutdown by some unknown reason.

2016-08-29 Thread Du, Jingcheng
Hi, Long GC pause delays the heartbeat from the region server to zookeeper, and make the connection between the region server and zookeeper timeout. Zookeeper deletes the znode of this region server after the connection is considered as expired, master receives the event and reassign regions own

RE: Hbase USERT

2016-07-28 Thread Du, Jingcheng
BEOHAR On Fri, Jul 29, 2016 at 9:10 AM, Du, Jingcheng wrote: > Hi Ankit, > > It seems like a memory issue in region servers. Did you monitor the > memory usage in region servers during the run? How about to increase the heap > size? > Do you get exceptions after the region serv

RE: Hbase USERT

2016-07-28 Thread Du, Jingcheng
Hi Ankit, It seems like a memory issue in region servers. Did you monitor the memory usage in region servers during the run? How about to increase the heap size? Do you get exceptions after the region server goes down? Mind sharing it here? Regards, Jingcheng -Original Message- From: Di

RE: DeleteColumn in KeyValue

2016-06-30 Thread Du, Jingcheng
In your case, the delete cell and put cell have the same timestamp and seqId if using bulkload. The delete cell can be read ahead of put cell, and the scan result will be null. You can try to use different timestamp for them? Regards, Jingcheng -Original Message- From: Krishna [mailto:

RE: [DISCUSS] Criteria for including MOB feature backport in branch-1

2016-03-03 Thread Du, Jingcheng
Sorry for the late chime in and thank you all for the discussion. I agree with Jon's suggestion, to make a new branch for MOB in branch-1.x, and backport it when it is definitely ready. I can start to work on that branch after the distributed mob compaction is finished. I think the distributed

RE: HBASE Table/schema design resource.

2016-02-03 Thread Du, Jingcheng
Hello Keith, Now HBase has a MOB feature in trunk (HBASE-11339), but it is not in any release branch. It might be merged back to 1.3 or 1.4 later. If your blob size is large (100KB-10MB), you could try this. In table design with this feature, you could store the blob in a single column family wi

RE: Build failed in Jenkins: HBase-Trunk_matrix ยป latest1.8,Hadoop #513

2015-12-03 Thread Du, Jingcheng
Hi, this issue is filed as HBASE-14907. Now the patch is available. It tried to ask for the HTableDescriptor after the table directory was removed (at that time this HTableDescriptor was not cached in memory). In the patch, it directly checks if the mob directory is there in fs instead of asking

Is it necessary to use an individual wait time for each procedure

2015-09-24 Thread Du, Jingcheng
Hi all, Currently HBaseAdmin.execProcedure uses a default wait time (SnapshotDescriptionUtils.DEFAULT_MAX_WAIT_TIME) for all procedure calls. Should we use an individual value for each procedure? I guess, we could add a return value for MasterProcedureManager.execProcedure which is a long t

RE: How to put a row with an older timestamp and get it?

2015-03-23 Thread Du, Jingcheng
Hi, Before you ran the compaction, did you flush the memstore for that table? I think the compaction only compacts the store files without memstore, just in my opinion. Regards, Jingcheng From: Ted Yu [mailto:yuzhih...@gmail.com] Sent: Monday, March 23, 2015 10:05 PM To: dev@hbase.apache.org S

RE: Backup HMasters will go down if the zk connection expires without recovery

2014-03-20 Thread Du, Jingcheng
cases. > In > > a production environment I would expect a site local strategy (could > > be daemontools etc.) for automatic service recovery, if that is desired. > > > > > > > > On Thu, Mar 20, 2014 at 12:43 AM, Du, Jingcheng > > > >wrote: > &

Backup HMasters will go down if the zk connection expires without recovery

2014-03-20 Thread Du, Jingcheng
Dear Devs, Now I encounter a problem in the HMaster. Currently I run multiple HMasters in a cluster. If the ZK connection of one of the backup HMasters expires, this backup HMaster will go down directly without recovering the ZK connection. I saw there were such code in the HMaster.abortNow(