Hi, guys

I reproduce this problem: https://issues.apache.org/jira/browse/KYLIN-3973

My env is Kylin 2.5.2 + HBase 2.2. After reading the code of hbase, I found the 
problem but don’t know the best method to resolve it.

in hbase coprocessor rpc, data pass to hbase client by 
CodedInputStream.newInstance(). in hbase 0.98 the input is byte[], and hbase 
2.2 is InputStream. When protobuf read the input data, byte[] will not trigger 
the size check in function refillBuffer. But in hbase 2.2, the inputStream will 
refill many times and finnaly hit the limit of sizeLimit(DEFAULT_SIZE_LIMIT). 
So the solution is enlarge this value.

Extra, DEFAULT_SIZE_LIMIT is 64M in protobuf 2.5.0, which is changed to 
Integer.MAX_VALUE in protobuf3.


A simple way to fix bug is useing input.setSizeLimit() to modify this limit. 
But coprocessor/endpoint/generated/CubeVisitProtos.java is an auto generated 
file.

  Another way is changing com.google<http://com.google>.protobuf:2.5.0 to a 
shaded package. Some other lib(such as hbase) choose this way.


I need your suggestion ~


Thank you.

Reply via email to