[
https://issues.apache.org/jira/browse/PHOENIX-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Samarth Jain resolved PHOENIX-2161.
-----------------------------------
Resolution: Invalid
> Can't change timeout
> --------------------
>
> Key: PHOENIX-2161
> URL: https://issues.apache.org/jira/browse/PHOENIX-2161
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.4.0
> Environment: Hadoop with Ambari 2.1.0
> Phoenix 4.4.0.2.3
> HBase 1.1.1.2.3
> HDFS 2.7.1.2.3
> Zookeeper 3.4.6.2.3
> Reporter: Adrià V.
> Labels: hbase, operation, phoenix, timeout
>
> Phoenix or HBase keeps throwing a timeout exception. I have tryed every
> configuration I could think about to increase it.
> Partial stacktrace:
> {quote}
> Caused by: java.io.IOException: Call to
> hdp-w-1.c.dks-hadoop.internal/10.240.2.235:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=43, waitTime=60001,
> operationTimeout=60000 expired.
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1242)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1210)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:213)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:369)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:343)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
> ... 4 more
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=43,
> waitTime=60001, operationTimeout=60000 expired.
> at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1184)
> ... 13 more
> {quote}
> The Phoenix (hbase-site.xml) properties:
> - phoenix.query.timeoutMs
> - phoenix.query.keepAliveMs
> I've tryed editing HBase config files and also setting config in Ambari with
> the next keys to increase the timeout with no success:
> - hbase.rpc.timeout
> - dfs.socket.timeout
> - dfs.client.socket-timeout
> - zookeeper.session.timeout
> Full stack trace:
> {quote}
> Error: Encountered exception in sub plan [0] execution. (state=,code=0)
> java.sql.SQLException: Encountered exception in sub plan [0] execution.
> at org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:157)
> at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:251)
> at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:241)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:240)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1250)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
> at sqlline.SqlLine.main(SqlLine.java:292)
> Caused by: org.apache.phoenix.exception.PhoenixIOException:
> org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36,
> exceptions:
> Mon Aug 03 16:47:06 UTC 2015, null, java.net.SocketTimeoutException:
> callTimeout=60000, callDuration=60303: row '' on table 'hive_post_topics' at
> region=hive_post_topics,,1438084107396.cdbdc246ff0b7dfed31d481e0bccd2b5.,
> hostname=hdp-w-1.c.dks-hadoop.internal,16020,1438619912282, seqNum=45322
> at
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
> at
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:542)
> at
> org.apache.phoenix.iterate.RoundRobinResultIterator.getIterators(RoundRobinResultIterator.java:176)
> at
> org.apache.phoenix.iterate.RoundRobinResultIterator.next(RoundRobinResultIterator.java:91)
> at org.apache.phoenix.join.HashCacheClient.serialize(HashCacheClient.java:106)
> at
> org.apache.phoenix.join.HashCacheClient.addHashCache(HashCacheClient.java:82)
> at
> org.apache.phoenix.execute.HashJoinPlan$HashSubPlan.execute(HashJoinPlan.java:339)
> at org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:136)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask.run(JobManager.java:172)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36,
> exceptions:
> Mon Aug 03 16:47:06 UTC 2015, null, java.net.SocketTimeoutException:
> callTimeout=60000, callDuration=60303: row '' on table 'hive_post_topics' at
> region=hive_post_topics,,1438084107396.cdbdc246ff0b7dfed31d481e0bccd2b5.,
> hostname=hdp-w-1.c.dks-hadoop.internal,16020,1438619912282, seqNum=45322
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:202)
> at
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:538)
> ... 11 more
> Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after
> attempts=36, exceptions:
> Mon Aug 03 16:47:06 UTC 2015, null, java.net.SocketTimeoutException:
> callTimeout=60000, callDuration=60303: row '' on table 'hive_post_topics' at
> region=hive_post_topics,,1438084107396.cdbdc246ff0b7dfed31d481e0bccd2b5.,
> hostname=hdp-w-1.c.dks-hadoop.internal,16020,1438619912282, seqNum=45322
> at
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
> at
> org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:56)
> at
> org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:104)
> at
> org.apache.phoenix.iterate.LookAheadResultIterator$1.advance(LookAheadResultIterator.java:47)
> at
> org.apache.phoenix.iterate.LookAheadResultIterator.init(LookAheadResultIterator.java:59)
> at
> org.apache.phoenix.iterate.LookAheadResultIterator.peek(LookAheadResultIterator.java:73)
> at
> org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:97)
> at
> org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:85)
> ... 5 more
> Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed
> after attempts=36, exceptions:
> Mon Aug 03 16:47:06 UTC 2015, null, java.net.SocketTimeoutException:
> callTimeout=60000, callDuration=60303: row '' on table 'hive_post_topics' at
> region=hive_post_topics,,1438084107396.cdbdc246ff0b7dfed31d481e0bccd2b5.,
> hostname=hdp-w-1.c.dks-hadoop.internal,16020,1438619912282, seqNum=45322
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:271)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:223)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
> at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
> at
> org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:403)
> at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:364)
> at
> org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:50)
> ... 11 more
> Caused by: java.net.SocketTimeoutException: callTimeout=60000,
> callDuration=60303: row '' on table 'hive_post_topics' at
> region=hive_post_topics,,1438084107396.cdbdc246ff0b7dfed31d481e0bccd2b5.,
> hostname=hdp-w-1.c.dks-hadoop.internal,16020,1438619912282, seqNum=45322
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159)
> at
> org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:64)
> ... 3 more
> Caused by: java.io.IOException: Call to
> hdp-w-1.c.dks-hadoop.internal/10.240.2.235:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=43, waitTime=60001,
> operationTimeout=60000 expired.
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1242)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1210)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:213)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:369)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:343)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
> ... 4 more
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=43,
> waitTime=60001, operationTimeout=60000 expired.
> at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1184)
> ... 13 more
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
