[
https://issues.apache.org/jira/browse/HADOOP-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499036#comment-17499036
]
Ayush Saxena commented on HADOOP-18143:
---------------------------------------
[~quapaw] Trying to understand the issue.
Seems the concurrency issue tends to occur when {{requestHeader}} is in
progress of being written, means half written and during that phase someone
calls {{getRequestHeader}} or say {{toString}} which indirectly calls
{{{}getRequestHeader{}}}. Can happen while debugging since the debugger calls
{{{}toString{}}}, or if someone is logging when the write is in progress.
May be making both {{getRequestHeader}} and {{writeTo}} synchronized or having
a lock as in the PR should solve this problem.
Seems this problem should surface even for ProtobufRpcEngine as well. I don't
see any reason which will prevent it there. If my understanding is correct.
I think this would surface during debugging only, if we remove the log
statement, right? I am not sure of the performance penalty what making a method
synchronized or taking lock can bring. But just a simple search on the internet
shows it would bring in some, if that is true, I don't think we can take that
cost, considering this is some commonly used code.
> toString method of RpcCall throws IllegalArgumentException
> ----------------------------------------------------------
>
> Key: HADOOP-18143
> URL: https://issues.apache.org/jira/browse/HADOOP-18143
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: András Győri
> Assignee: András Győri
> Priority: Critical
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We have observed breaking tests such as TestApplicationACLs. We have located
> the root cause, which is HADOOP-18082. It seems that there is a concurrency
> issue within ProtobufRpcEngine2. When using a debugger, the missing fields
> are there, hence the suspicion of concurrency problem. The stack trace:
> {noformat}
> java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:244)
> at
> org.apache.hadoop.ipc.RpcWritable$ProtobufWrapper.readFrom(RpcWritable.java:131)
> at org.apache.hadoop.ipc.RpcWritable$Buffer.getValue(RpcWritable.java:232)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$RpcProtobufRequest.getRequestHeader(ProtobufRpcEngine2.java:645)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$RpcProtobufRequest.toString(ProtobufRpcEngine2.java:663)
> at java.lang.String.valueOf(String.java:3425)
> at java.lang.StringBuilder.append(StringBuilder.java:516)
> at org.apache.hadoop.ipc.Server$RpcCall.toString(Server.java:1328)
> at java.lang.String.valueOf(String.java:3425)
> at java.lang.StringBuilder.append(StringBuilder.java:516)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3097){noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
