[jira] [Commented] (HDFS-8409) HDFS client RPC call throws "java.lang.IllegalStateException"

2019-04-10 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814873#comment-16814873
 ] 

Hadoop QA commented on HDFS-8409:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m  
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 
 8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  2m 
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
16m 32s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
49s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m 
36s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
3m  1s{color} | {color:orange} root: The patch generated 2 new + 139 unchanged 
- 0 fixed = 141 total (was 139) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  2m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
10m 59s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m  
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  8m 
12s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 42s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
47s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}185m 18s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.web.TestWebHdfsTimeouts |
|   | hadoop.hdfs.tools.TestDFSZKFailoverController |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-8409 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12735456/HDFS-8409.003.patch |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux 48e13cf7b9c5 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0e770a6 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| 

[jira] [Commented] (HDFS-8409) HDFS client RPC call throws "java.lang.IllegalStateException"

2019-04-10 Thread Xue Liu (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814726#comment-16814726
 ] 

Xue Liu commented on HDFS-8409:
---

We are also seeing a lot of this in our prod cluster. I think that this 
exception will be thrown if callId is not reset and then retry happens. Can 
someone take a further look at this? Thanks!

> HDFS client RPC call throws "java.lang.IllegalStateException"
> -
>
> Key: HDFS-8409
> URL: https://issues.apache.org/jira/browse/HDFS-8409
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs-client
>Reporter: Juan Yu
>Assignee: Juan Yu
>Priority: Major
> Attachments: HDFS-8409.001.patch, HDFS-8409.002.patch, 
> HDFS-8409.003.patch
>
>
> When the HDFS client RPC calls need to retry, it sometimes throws 
> "java.lang.IllegalStateException" and retry is aborted and cause the client 
> call will fail.
> {code}
> Caused by: java.lang.IllegalStateException
>   at 
> com.google.common.base.Preconditions.checkState(Preconditions.java:129)
>   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
>   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
>   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
>   at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
>   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
> {code}
> Here is the check that throws exception
> {code}
>   public static void setCallIdAndRetryCount(int cid, int rc) {
>   ...
>   Preconditions.checkState(callId.get() == null);
>   }
> {code}
> The RetryInvocationHandler tries to call it with not null callId and causes 
> exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Commented] (HDFS-8409) HDFS client RPC call throws "java.lang.IllegalStateException"

2017-03-02 Thread Yongtao Yang (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892062#comment-15892062
 ] 

Yongtao Yang commented on HDFS-8409:


I found a clue that may be useful to this problem. 
DFSClient#namenode is a proxy whose InvocationHandler is 
RetryInvocationHandler. If namenode.toString() or namenode.hashCode() or some 
other non-rpc methods are called, then the next real rpc method will failed at 
setCallIdAndRetryCount(). The cause of this is that toString() is also 
forwarded to RetryInvocationHandler.invoke() and setCallIdAndRetryCount() is 
executed, then ipc.Client#callId.get() will not be null any more. For real rpc 
methods, ipc.Client#callId will be set to null when creating a Call 
instance(org.apache.hadoop.ipc.Client.Call.Call(RpcKind, Writable)), but for 
toString() or hashCode(), it can't reach the Call constructor, so 
ipc.Client#callId will never recover to be null and the next method(whether it 
is a real rpc or not) will fail.

> HDFS client RPC call throws "java.lang.IllegalStateException"
> -
>
> Key: HDFS-8409
> URL: https://issues.apache.org/jira/browse/HDFS-8409
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs-client
>Reporter: Juan Yu
>Assignee: Juan Yu
> Attachments: HDFS-8409.001.patch, HDFS-8409.002.patch, 
> HDFS-8409.003.patch
>
>
> When the HDFS client RPC calls need to retry, it sometimes throws 
> "java.lang.IllegalStateException" and retry is aborted and cause the client 
> call will fail.
> {code}
> Caused by: java.lang.IllegalStateException
>   at 
> com.google.common.base.Preconditions.checkState(Preconditions.java:129)
>   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
>   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
>   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
>   at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
>   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
> {code}
> Here is the check that throws exception
> {code}
>   public static void setCallIdAndRetryCount(int cid, int rc) {
>   ...
>   Preconditions.checkState(callId.get() == null);
>   }
> {code}
> The RetryInvocationHandler tries to call it with not null callId and causes 
> exception.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Commented] (HDFS-8409) HDFS client RPC call throws "java.lang.IllegalStateException"

2017-01-19 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15830064#comment-15830064
 ] 

Hadoop QA commented on HDFS-8409:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  2m  
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
33s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m 
12s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
1m 40s{color} | {color:orange} root: The patch generated 2 new + 144 unchanged 
- 0 fixed = 146 total (was 144) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  2m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  7m 
38s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 92m 12s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
45s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 38s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker |
|   | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HDFS-8409 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12735456/HDFS-8409.003.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux 9f698a75d354 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 85e4961 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HDFS-Build/18211/artifact/patchprocess/diff-checkstyle-root.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/18211/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/18211/testReport/ |
| 

[jira] [Commented] (HDFS-8409) HDFS client RPC call throws "java.lang.IllegalStateException"

2017-01-19 Thread Zhaofei Meng (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829812#comment-15829812
 ] 

Zhaofei Meng commented on HDFS-8409:


2017-01-16 06:00:50,236 ERROR metastore.RetryingHMSHandler 
(RetryingHMSHandler.java:invoke(155)) - 
MetaException(message:java.lang.IllegalStateException) at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:5568)
 at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1503)
 at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606) at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
 at com.sun.proxy.$Proxy10.create_table_with_environment_context(Unknown 
Source) at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:9817)
 at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:9801)
 at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
 at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
 at java.security.AccessController.doPrivileged(Native Method) at 
javax.security.auth.Subject.doAs(Subject.java:415) at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
 at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
 at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
at java.lang.Thread.run(Thread.java:745) Caused by: 
java.lang.IllegalStateException at 
com.google.common.base.Preconditions.checkState(Preconditions.java:129) at 
org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116) at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
 at com.sun.proxy.$Proxy15.getFileInfo(Unknown Source) at 
org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1701) at 
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1124)
 at 
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
 at 
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
 at 
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1120)
 at org.apache.hadoop.hive.metastore.Warehouse.isDir(Warehouse.java:475) at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1430)
 at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1489)
 ... 18 more

> HDFS client RPC call throws "java.lang.IllegalStateException"
> -
>
> Key: HDFS-8409
> URL: https://issues.apache.org/jira/browse/HDFS-8409
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs-client
>Reporter: Juan Yu
>Assignee: Juan Yu
> Attachments: HDFS-8409.001.patch, HDFS-8409.002.patch, 
> HDFS-8409.003.patch
>
>
> When the HDFS client RPC calls need to retry, it sometimes throws 
> "java.lang.IllegalStateException" and retry is aborted and cause the client 
> call will fail.
> {code}
> Caused by: java.lang.IllegalStateException
>   at 
> com.google.common.base.Preconditions.checkState(Preconditions.java:129)
>   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
>   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
>   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
>   at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
>   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
> {code}
> Here is the check that throws exception
> {code}
>   public static void setCallIdAndRetryCount(int cid, int rc) {
>   ...
>  

[jira] [Commented] (HDFS-8409) HDFS client RPC call throws java.lang.IllegalStateException

2015-06-01 Thread Andrew Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568011#comment-14568011
 ] 

Andrew Wang commented on HDFS-8409:
---

I guess I'm confused where these exceptions are coming from. This error happens 
if we call {{setCallIdAndRetryCount}} twice without an intervening creation of 
a {{Call}}. This means an exception somewhere in {{invokeMethod}} before the 
{{Call}} is created. I looked at {{ProtobufRpcEngine#invoke}} and I don't see 
any non-fatal exceptions that would be fixed by a retry. Am I off-base here?

Do you have more logging from when this happened?

 HDFS client RPC call throws java.lang.IllegalStateException
 -

 Key: HDFS-8409
 URL: https://issues.apache.org/jira/browse/HDFS-8409
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs-client
Reporter: Juan Yu
Assignee: Juan Yu
 Attachments: HDFS-8409.001.patch, HDFS-8409.002.patch, 
 HDFS-8409.003.patch


 When the HDFS client RPC calls need to retry, it sometimes throws 
 java.lang.IllegalStateException and retry is aborted and cause the client 
 call will fail.
 {code}
 Caused by: java.lang.IllegalStateException
   at 
 com.google.common.base.Preconditions.checkState(Preconditions.java:129)
   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
   at 
 org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
 {code}
 Here is the check that throws exception
 {code}
   public static void setCallIdAndRetryCount(int cid, int rc) {
   ...
   Preconditions.checkState(callId.get() == null);
   }
 {code}
 The RetryInvocationHandler tries to call it with not null callId and causes 
 exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8409) HDFS client RPC call throws java.lang.IllegalStateException

2015-05-29 Thread Juan Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14565649#comment-14565649
 ] 

Juan Yu commented on HDFS-8409:
---

If it happens at retry, not the initial call. For example, the initial call 
gets an exception after call object is created and sent. so it needs retry, but 
during retry, somehow it gets exception again and this time even before call 
object (should be same callId as the initial call) is created.
In my patch, I added a test to simulate it, does it make sense?

 HDFS client RPC call throws java.lang.IllegalStateException
 -

 Key: HDFS-8409
 URL: https://issues.apache.org/jira/browse/HDFS-8409
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs-client
Reporter: Juan Yu
Assignee: Juan Yu
 Attachments: HDFS-8409.001.patch, HDFS-8409.002.patch, 
 HDFS-8409.003.patch


 When the HDFS client RPC calls need to retry, it sometimes throws 
 java.lang.IllegalStateException and retry is aborted and cause the client 
 call will fail.
 {code}
 Caused by: java.lang.IllegalStateException
   at 
 com.google.common.base.Preconditions.checkState(Preconditions.java:129)
   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
   at 
 org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
 {code}
 Here is the check that throws exception
 {code}
   public static void setCallIdAndRetryCount(int cid, int rc) {
   ...
   Preconditions.checkState(callId.get() == null);
   }
 {code}
 The RetryInvocationHandler tries to call it with not null callId and causes 
 exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8409) HDFS client RPC call throws java.lang.IllegalStateException

2015-05-29 Thread Andrew Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14565605#comment-14565605
 ] 

Andrew Wang commented on HDFS-8409:
---

Hey Juan, when would an exception before creation of a Call object not be a 
fatal error?

 HDFS client RPC call throws java.lang.IllegalStateException
 -

 Key: HDFS-8409
 URL: https://issues.apache.org/jira/browse/HDFS-8409
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs-client
Reporter: Juan Yu
Assignee: Juan Yu
 Attachments: HDFS-8409.001.patch, HDFS-8409.002.patch, 
 HDFS-8409.003.patch


 When the HDFS client RPC calls need to retry, it sometimes throws 
 java.lang.IllegalStateException and retry is aborted and cause the client 
 call will fail.
 {code}
 Caused by: java.lang.IllegalStateException
   at 
 com.google.common.base.Preconditions.checkState(Preconditions.java:129)
   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
   at 
 org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
 {code}
 Here is the check that throws exception
 {code}
   public static void setCallIdAndRetryCount(int cid, int rc) {
   ...
   Preconditions.checkState(callId.get() == null);
   }
 {code}
 The RetryInvocationHandler tries to call it with not null callId and causes 
 exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8409) HDFS client RPC call throws java.lang.IllegalStateException

2015-05-26 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560345#comment-14560345
 ] 

Hadoop QA commented on HDFS-8409:
-

\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  14m 49s | Pre-patch trunk compilation is 
healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any 
@author tags. |
| {color:green}+1{color} | tests included |   0m  0s | The patch appears to 
include 1 new or modified test files. |
| {color:green}+1{color} | javac |   7m 28s | There were no new javac warning 
messages. |
| {color:green}+1{color} | javadoc |   9m 39s | There were no new javadoc 
warning messages. |
| {color:green}+1{color} | release audit |   0m 23s | The applied patch does 
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle |   3m 22s | There were no new checkstyle 
issues. |
| {color:green}+1{color} | whitespace |   0m  0s | The patch has no lines that 
end in whitespace. |
| {color:green}+1{color} | install |   1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 32s | The patch built with 
eclipse:eclipse. |
| {color:green}+1{color} | findbugs |   4m 45s | The patch does not introduce 
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | common tests |  23m  4s | Tests passed in 
hadoop-common. |
| {color:green}+1{color} | hdfs tests | 162m 40s | Tests passed in hadoop-hdfs. 
|
| | | 228m 21s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12735456/HDFS-8409.003.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / cdbd66b |
| hadoop-common test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11134/artifact/patchprocess/testrun_hadoop-common.txt
 |
| hadoop-hdfs test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11134/artifact/patchprocess/testrun_hadoop-hdfs.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11134/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11134/console |


This message was automatically generated.

 HDFS client RPC call throws java.lang.IllegalStateException
 -

 Key: HDFS-8409
 URL: https://issues.apache.org/jira/browse/HDFS-8409
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs-client
Reporter: Juan Yu
Assignee: Juan Yu
 Attachments: HDFS-8409.001.patch, HDFS-8409.002.patch, 
 HDFS-8409.003.patch


 When the HDFS client RPC calls need to retry, it sometimes throws 
 java.lang.IllegalStateException and retry is aborted and cause the client 
 call will fail.
 {code}
 Caused by: java.lang.IllegalStateException
   at 
 com.google.common.base.Preconditions.checkState(Preconditions.java:129)
   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
   at 
 org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
 {code}
 Here is the check that throws exception
 {code}
   public static void setCallIdAndRetryCount(int cid, int rc) {
   ...
   Preconditions.checkState(callId.get() == null);
   }
 {code}
 The RetryInvocationHandler tries to call it with not null callId and causes 
 exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8409) HDFS client RPC call throws java.lang.IllegalStateException

2015-05-23 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14557501#comment-14557501
 ] 

Hadoop QA commented on HDFS-8409:
-

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  14m 34s | Pre-patch trunk compilation is 
healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any 
@author tags. |
| {color:green}+1{color} | tests included |   0m  0s | The patch appears to 
include 1 new or modified test files. |
| {color:green}+1{color} | javac |   7m 31s | There were no new javac warning 
messages. |
| {color:green}+1{color} | javadoc |   9m 37s | There were no new javadoc 
warning messages. |
| {color:green}+1{color} | release audit |   0m 22s | The applied patch does 
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle |   1m 54s | There were no new checkstyle 
issues. |
| {color:red}-1{color} | whitespace |   0m  0s | The patch has 2  line(s) that 
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install |   1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 33s | The patch built with 
eclipse:eclipse. |
| {color:green}+1{color} | findbugs |   4m 43s | The patch does not introduce 
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | common tests |  22m 39s | Tests passed in 
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 161m 36s | Tests failed in hadoop-hdfs. |
| | | 225m  5s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.datanode.TestDataNodeMetrics |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12735023/HDFS-8409.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 446d515 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HDFS-Build/5/artifact/patchprocess/whitespace.txt
 |
| hadoop-common test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/5/artifact/patchprocess/testrun_hadoop-common.txt
 |
| hadoop-hdfs test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/5/artifact/patchprocess/testrun_hadoop-hdfs.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/5/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/5/console |


This message was automatically generated.

 HDFS client RPC call throws java.lang.IllegalStateException
 -

 Key: HDFS-8409
 URL: https://issues.apache.org/jira/browse/HDFS-8409
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs-client
Reporter: Juan Yu
Assignee: Juan Yu
 Attachments: HDFS-8409.001.patch, HDFS-8409.002.patch


 When the HDFS client RPC calls need to retry, it sometimes throws 
 java.lang.IllegalStateException and retry is aborted and cause the client 
 call will fail.
 {code}
 Caused by: java.lang.IllegalStateException
   at 
 com.google.common.base.Preconditions.checkState(Preconditions.java:129)
   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
   at 
 org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
 {code}
 Here is the check that throws exception
 {code}
   public static void setCallIdAndRetryCount(int cid, int rc) {
   ...
   Preconditions.checkState(callId.get() == null);
   }
 {code}
 The RetryInvocationHandler tries to call it with not null callId and causes 
 exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8409) HDFS client RPC call throws java.lang.IllegalStateException

2015-05-23 Thread Juan Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14557409#comment-14557409
 ] 

Juan Yu commented on HDFS-8409:
---

The test failure is because {code}java.lang.NoClassDefFoundError: 
org/apache/hadoop/tracing/SpanReceiverHost {code}
that's not related with my change.
I ran those tests locally and both passed.

 HDFS client RPC call throws java.lang.IllegalStateException
 -

 Key: HDFS-8409
 URL: https://issues.apache.org/jira/browse/HDFS-8409
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs-client
Reporter: Juan Yu
Assignee: Juan Yu
 Attachments: HDFS-8409.001.patch


 When the HDFS client RPC calls need to retry, it sometimes throws 
 java.lang.IllegalStateException and retry is aborted and cause the client 
 call will fail.
 {code}
 Caused by: java.lang.IllegalStateException
   at 
 com.google.common.base.Preconditions.checkState(Preconditions.java:129)
   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
   at 
 org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
 {code}
 Here is the check that throws exception
 {code}
   public static void setCallIdAndRetryCount(int cid, int rc) {
   ...
   Preconditions.checkState(callId.get() == null);
   }
 {code}
 The RetryInvocationHandler tries to call it with not null callId and causes 
 exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8409) HDFS client RPC call throws java.lang.IllegalStateException

2015-05-22 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555802#comment-14555802
 ] 

Hadoop QA commented on HDFS-8409:
-

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  14m 32s | Pre-patch trunk compilation is 
healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any 
@author tags. |
| {color:green}+1{color} | tests included |   0m  0s | The patch appears to 
include 1 new or modified test files. |
| {color:green}+1{color} | javac |   7m 31s | There were no new javac warning 
messages. |
| {color:green}+1{color} | javadoc |   9m 40s | There were no new javadoc 
warning messages. |
| {color:green}+1{color} | release audit |   0m 22s | The applied patch does 
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle |   3m 18s | There were no new checkstyle 
issues. |
| {color:red}-1{color} | whitespace |   0m  0s | The patch has 2  line(s) that 
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install |   1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 32s | The patch built with 
eclipse:eclipse. |
| {color:green}+1{color} | findbugs |   4m 42s | The patch does not introduce 
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | common tests |  23m  5s | Tests passed in 
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 163m 56s | Tests failed in hadoop-hdfs. |
| | | 229m 17s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | 
hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations |
| Timed out tests | org.apache.hadoop.hdfs.server.mover.TestMover |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12734734/HDFS-8409.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / cf2b569 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11097/artifact/patchprocess/whitespace.txt
 |
| hadoop-common test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11097/artifact/patchprocess/testrun_hadoop-common.txt
 |
| hadoop-hdfs test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11097/artifact/patchprocess/testrun_hadoop-hdfs.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11097/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11097/console |


This message was automatically generated.

 HDFS client RPC call throws java.lang.IllegalStateException
 -

 Key: HDFS-8409
 URL: https://issues.apache.org/jira/browse/HDFS-8409
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs-client
Reporter: Juan Yu
Assignee: Juan Yu
 Attachments: HDFS-8409.001.patch


 When the HDFS client RPC calls need to retry, it sometimes throws 
 java.lang.IllegalStateException and retry is aborted and cause the client 
 call will fail.
 {code}
 Caused by: java.lang.IllegalStateException
   at 
 com.google.common.base.Preconditions.checkState(Preconditions.java:129)
   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
   at 
 org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
 {code}
 Here is the check that throws exception
 {code}
   public static void setCallIdAndRetryCount(int cid, int rc) {
   ...
   Preconditions.checkState(callId.get() == null);
   }
 {code}
 The RetryInvocationHandler tries to call it with not null callId and causes 
 exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8409) HDFS client RPC call throws java.lang.IllegalStateException

2015-05-21 Thread Juan Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1441#comment-1441
 ] 

Juan Yu commented on HDFS-8409:
---

Client.callId is only reset to null when a new RPC call object created.
During retry, we keep the same callId. if exception happened before RPC call 
object creation, Client.callId won't be reset to null and 
{code}Preconditions.checkState(callId.get() == null);{code} check will fail and 
cause RPC retry fail.
We should accept not null callId as long as it's the same as cid.

 HDFS client RPC call throws java.lang.IllegalStateException
 -

 Key: HDFS-8409
 URL: https://issues.apache.org/jira/browse/HDFS-8409
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs-client
Reporter: Juan Yu
Assignee: Juan Yu
 Attachments: HDFS-8409.001.patch


 When the HDFS client RPC calls need to retry, it sometimes throws 
 java.lang.IllegalStateException and retry is aborted and cause the client 
 call will fail.
 {code}
 Caused by: java.lang.IllegalStateException
   at 
 com.google.common.base.Preconditions.checkState(Preconditions.java:129)
   at org.apache.hadoop.ipc.Client.setCallIdAndRetryCount(Client.java:116)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:99)
   at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1912)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1089)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
   at 
 org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
 {code}
 Here is the check that throws exception
 {code}
   public static void setCallIdAndRetryCount(int cid, int rc) {
   ...
   Preconditions.checkState(callId.get() == null);
   }
 {code}
 The RetryInvocationHandler tries to call it with not null callId and causes 
 exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)