actually, there is only one huge value in the hbase cell which large than 100M, maybe it's not a good idea to store such huge value in hbase.
Any suggestions to store this huge objects? 在 16/10/19 下午8:16, 吴国泉wgq 写道: > hi biodata: > > you can try “ scan.setbatch()” or other filter to limit the number > of column returned. > > This is because: There is a very large row in your table,when you try > to retrieve it, OOM will happen. > > As I can see, There is no other method to solve this problem. > Default,every read operation, hbase will retrieve an entire row if you don’t > limit the col。 > > Don’t insert very large row. you can delete it . But don’t forget > trigger the major compaction after you delete the row. > > > > 吴国泉 wgq.wu > Post: DBA Hbase > Email: [email protected]<http://qunar.com> > Tel: 13051697997 > Adr: 中国电子大厦17层 > > > > 在 2016年10月19日，下午7:43，big data > <[email protected]<mailto:[email protected]>> 写道： > > I've adjusted the jvm xmx in hbase-env.xml, now in hbase shell, count > runs well. > > But java client still crashes because : > > Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol > message was too large. May be malicious. Use > CodedInputStream.setSizeLimit() to increase the size limit. > > I've browsed apache's jira, but still don't know how to call > CodedInputStream.setSizeLimit() in client side. > > > 在 16/10/19 下午6:38, Jean-Marc Spaggiari 写道: > Interesting. Can you bump the client heap size? How much do you have for > the client? > > JMS > > 2016-10-19 3:50 GMT-04:00 big data > <[email protected]<mailto:[email protected]>>: > > Dear all, > > I've a hbase table, one row has a huge keyvalue, about 100M size. > > When I execute count table in hbase shell, hbase crash to bash, and > display error like this: > > hbase(main):005:0> count 'table', CACHE=>10000 > java.lang.OutOfMemoryError: Java heap space > Dumping heap to java_pid17735.hprof ... > Unable to create java_pid17735.hprof: 权限不够 > # > # java.lang.OutOfMemoryError: Java heap space > # -XX:OnOutOfMemoryError="kill -9 %p" > # Executing /bin/sh -c "kill -9 17735"... > 已杀死 > > Meanwhile, my java client execute get operation from this table, it > hangs for a long time. > > How can I adjust some parameters to support huge keyvalue? > > > > > > > > 安全提示：本邮件非QUNAR内部邮件，请注意保护个人及公司信息安全，如有索取帐号密码等可疑情况请向 secteam发送邮件 >
