[
https://issues.apache.org/jira/browse/HBASE-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17551696#comment-17551696
]
Andrew Kyle Purtell edited comment on HBASE-27097 at 6/8/22 4:30 PM:
---------------------------------------------------------------------
bq. Also, compatibility wise, would it be ok to remove it entirely on a minor
release?
Correct, that is an issue which is why I said this:
bq. I realize this is a technical violation of the operational compatibility
guideline for a minor release but unless a volunteer steps forward to fix it,
there is no choice.
I will note that the last significant change to SimpleRpcServer was in 2017,
that was 5 years ago, and nobody has showed up until now, technically not even
now as I noticed this problem but have no intention to use SimpleRpcServer, so
how important is SimpleRpcServer really? Surely this problem would have been
noticed and fixed long before this if anyone was actually using it. So removal
is, of course, fine as a practical matter.
That said, we can choose to fix this with intention, deciding that
SimpleRpcServer should be a viable alternative or fallback to NettyRpcServer.
That would be fine too.
bq. Looks like SimpleRpcServer is completely broken in 2.5.
The issue developed in branch-2 and master, and when 2.5 was forked from
branch-2, then also in branch-2.5.
bq. among our customers running deployments based out of 2.2. In such cases,
our immediate workaround in the absence of a code fix is to switch to
SimpleRpcServer.
So just to confirm, you have customers successfully using SimpleRpcServer with
2.2.
was (Author: apurtell):
bq. Also, compatibility wise, would it be ok to remove it entirely on a minor
release?
Correct, that is an issue which is why I said this:
bq. I realize this is a technical violation of the operational compatibility
guideline for a minor release but unless a volunteer steps forward to fix it,
there is no choice.
I will note that the last significant change to SimpleRpcServer was in 2017,
that was 5 years ago, and nobody has showed up until now, technically not even
now as I noticed this problem but have no intention to use SimpleRpcServer, so
how important is SimpleRpcServer really? Surely this problem would have been
noticed and fixed long before this if anyone was actually using it. So removal
is, of course, fine as a practical matter.
That said, we can choose to fix this with intention, deciding that
SimpleRpcServer should be a viable alternative or fallback to NettyRpcServer.
That would be fine too.
> SimpleRpcServer is broken
> -------------------------
>
> Key: HBASE-27097
> URL: https://issues.apache.org/jira/browse/HBASE-27097
> Project: HBase
> Issue Type: Bug
> Components: rpc
> Affects Versions: 2.5.0
> Reporter: Andrew Kyle Purtell
> Priority: Major
>
> Concerns about SimpleRpcServer are not new, and not new to 2.5. @chenxu
> noticed a problem on HBASE-23917 back in 2020. After some simple evaluations
> it seems quite broken.
> When I run an async version of ITLCC against a 2.5.0 cluster configured with
> hbase.rpc.server.impl=SimpleRpcServer, the client almost immediately stalls
> because there are too many in flight requests. The logic to pause with too
> many in flight requests is my own. That's not important. Looking at the
> server logs it is apparent that SimpleRpcServer is quite broken. Handlers
> suffer frequent protobuf parse errors and do not properly return responses to
> the client. This is what stalls my test client. Rather quickly all available
> request slots are full of requests that will have to time out on the client
> side.
> Exceptions have three patterns but they all have in common
> SimpleServerRpcConnection#process. It seems likely the root cause is
> mismatched expectations or bugs in connection buffer handling in
> SimpleRpcServer/SimpleServerRpcConnection versus downstream classes that
> process and parse the buffers. It also seems likely that changes were made to
> downstream classes like ServerRpcConnection expecting NettyRpcServer's
> particulars without updating SimpleServerRpcConnection and/or
> SimpleRpcServer. That said, this is just a superficial analysis.
> 1) "Protocol message end-group tag did not match expected tag"
> {noformat}
> 2022-06-07T16:44:04,625 WARN
> [Reader=5,bindAddress=buildbox.localdomain,port=8120] ipc.RpcServer:
> /127.0.1.1:8120 is unable to read call parameter from client 127.0.0.1
> org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException:
> Protocol message end-group tag did not match expected tag.
> at
> org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:129)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.checkLastTagWas(CodedInputStream.java:4034)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4275)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10520)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10464)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12251)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12245)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9981)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9910)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14097)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14091)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14251)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14190)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15304)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15298)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14860)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14651)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:420)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:317)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.mergeFrom(ProtobufUtil.java:2638)
> ~[hbase-client-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.ServerRpcConnection.processRequest(ServerRpcConnection.java:644)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:444)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.process(SimpleServerRpcConnection.java:285)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.readAndProcess(SimpleServerRpcConnection.java:251)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.doRead(SimpleRpcServer.java:318)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:180)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:153)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> {noformat}
> 2) "Protocol message tag had invalid wire type."
> {noformat}
> 2022-06-07T16:44:04,705 WARN
> [Reader=6,bindAddress=buildbox.localdomain,port=8120] ipc.RpcServer:
> /127.0.1.1:8120 is unable to read call parameter from client 127.0.0.1
> org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException:
> Protocol message tag had invalid wire type.
> at
> org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:134)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:527)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.GeneratedMessageV3.parseUnknownField(GeneratedMessageV3.java:320)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue.<init>(ClientProtos.java:10700)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue.<init>(ClientProtos.java:10620)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue$1.parsePartialFrom(ClientProtos.java:11481)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue$1.parsePartialFrom(ClientProtos.java:11475)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10520)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10464)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12251)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12245)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9981)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9910)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14097)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14091)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14251)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14190)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15304)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15298)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14860)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14651)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:420)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:317)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.mergeFrom(ProtobufUtil.java:2638)
> ~[hbase-client-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.ServerRpcConnection.processRequest(ServerRpcConnection.java:644)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:444)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.process(SimpleServerRpcConnection.java:285)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.readAndProcess(SimpleServerRpcConnection.java:251)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.doRead(SimpleRpcServer.java:318)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:180)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:153)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ~[?:?]
> at java.lang.Thread.run(Thread.java:829) ~[?:?]
> {noformat}
> 3) "While parsing a protocol message, the input ended unexpectedly in the
> middle of a field."
> {noformat}
> 2022-06-07T16:44:04,885 WARN
> [Reader=9,bindAddress=buildbox.localdomain,port=8120] ipc.RpcServer:
> /127.0.1.1:8120 is unable to read call parameter from client 127.0.0.1
> org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException:
> While parsing a protocol message, the input ended unexpectedly in the middle
> of a field. This could mean either that the input has been truncated or that
> an embedded message misreported its own length.
> at
> org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException.truncatedMessage(InvalidProtocolBufferException.java:107)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readRawLittleEndian64(CodedInputStream.java:4478)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readFixed64(CodedInputStream.java:4167)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:511)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.GeneratedMessageV3.parseUnknownField(GeneratedMessageV3.java:320)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue.<init>(ClientProtos.java:10700)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue.<init>(ClientProtos.java:10620)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue$1.parsePartialFrom(ClientProtos.java:11481)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue$1.parsePartialFrom(ClientProtos.java:11475)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10520)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10464)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12251)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12245)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9981)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9910)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14097)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14091)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14251)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14190)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15304)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15298)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14860)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14651)
> ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:420)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:317)
> ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0]
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.mergeFrom(ProtobufUtil.java:2638)
> ~[hbase-client-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.ServerRpcConnection.processRequest(ServerRpcConnection.java:644)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:444)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.process(SimpleServerRpcConnection.java:285)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.readAndProcess(SimpleServerRpcConnection.java:251)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.doRead(SimpleRpcServer.java:318)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:180)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:153)
> ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ~[?:?]
> at java.lang.Thread.run(Thread.java:829) ~[?:?]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
