The snapshot didn't go through. Can you put it on some website and give us the URL ?
Cheers On May 22, 2014, at 7:38 PM, jingych <jing...@neusoft.com> wrote: > Hi, JM & everyone! > > Thanks for your reply! > > My java client, use the HTable#put(Put) method with 2M buffer to commit rows. > While the client running for a while, the jvisualvm shows the htable threads > increasing and the heap size increased too. > Here is the snapshot: > > > i research the implementation of the HTable#put(Put). > I found the doPut operation is the async, and when one row success, this will > return. > > So, I guess when the HBase server is slower to response the put request. > The hbase client will accumulate so many rows. > And on the other hand, when the put operation failed, the hbase client will > retry with the recursion submit method. > > I think the above two is the reason caused the OOME. > > > PS: > What do you mean It's not cdh5 specific? I found it from the website > http://archive.cloudera.com/cdh5/cdh/5/hbase-0.96.1.1-cdh5.0.0/ > Where can i get the CDH5 source code ? > CDH5 means hbase 0.96.2? > I'm confused with the CDH5, Apache, Phoenix ... > What's the relationship among them. > > Thanks a lot! > > Best regards! > > Jingych > > From: Jean-Marc Spaggiari > Date: 2014-05-23 09:33 > To: user; jingych > Subject: Re: CDH5 hbase client outofmemory > Hi Jingych, > > This is the HBase 0.96.2 code. Not CDH5 specific. Do you have more details on > you OOME? Did you figured why it occured? > > JM > > > 2014-05-22 1:20 GMT-04:00 jingych <jing...@neusoft.com>: >> Hello, everyone! >> >> I found the CDH5 hbase client swallowed the Outofmemory exception. >> >> It didn't throw out, cause the program couldn't make the correct response to >> process the OOM. >> >> Is't good to catch the client OOM? Why? >> >> The CDH5 hbase client process the throwable code: >> private boolean manageError(int originalIndex, Row row, boolean canRetry, >> Throwable throwable, HRegionLocation location) >> { >> if (canRetry && throwable != null && throwable instanceof >> DoNotRetryIOException) { >> canRetry = false; >> } >> >> byte[] region = null; >> if (canRetry && callback != null) { >> region = location == null ? null : >> location.getRegionInfo().getEncodedNameAsBytes(); >> canRetry = callback.retriableFailure(originalIndex, row, region, >> throwable); >> } >> >> if (!canRetry) { >> if (callback != null) { >> if (region == null && location != null) { >> region = location.getRegionInfo().getEncodedNameAsBytes(); >> } >> callback.failure(originalIndex, region, row, throwable); >> } >> errors.add(throwable, row, location); >> this.hasError.set(true); >> } >> >> return canRetry; >> } >> >> This will treat the OOM as the canRetry exception. >> >> Thanks! >> >> Best Regards! >> >> >> >> >> jingych >> --------------------------------------------------------------------------------------------------- >> Confidentiality Notice: The information contained in this e-mail and any >> accompanying attachment(s) >> is intended only for the use of the intended recipient and may be >> confidential and/or privileged of >> Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader >> of this communication is >> not the intended recipient, unauthorized use, forwarding, printing, >> storing, disclosure or copying >> is strictly prohibited, and may be unlawful.If you have received this >> communication in error,please >> immediately notify the sender by return e-mail, and delete the original >> message and all copies from >> your system. Thank you. >> --------------------------------------------------------------------------------------------------- > > --------------------------------------------------------------------------------------------------- > Confidentiality Notice: The information contained in this e-mail and any > accompanying attachment(s) > is intended only for the use of the intended recipient and may be > confidential and/or privileged of > Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of > this communication is > not the intended recipient, unauthorized use, forwarding, printing, storing, > disclosure or copying > is strictly prohibited, and may be unlawful.If you have received this > communication in error,please > immediately notify the sender by return e-mail, and delete the original > message and all copies from > your system. Thank you. > ---------------------------------------------------------------------------------------------------
