Keith Chen created KYLIN-3329: --------------------------------- Summary: Step : Convert Cuboid Data to HFile , Call Hbase TimeOut , But status still running Key: KYLIN-3329 URL: https://issues.apache.org/jira/browse/KYLIN-3329 Project: Kylin Issue Type: Bug Reporter: Keith Chen
Step : Convert Cuboid Data to HFile, Call Hbase TimeOut.But status not change, it still running. 2018-04-02 04:07:49,059 ERROR [Scheduler 107735485 Job 09d4c09b-7a53-43f9-9229-f1951c90f6ef-576] dao.ExecutableDao:221 : error update job output id:09d4c09b-7a53-43f9-9229-f1951c90f6ef-14 org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions: Mon Apr 02 04:07:49 GMT+08:00 2018, RpcRetryingCaller\{globalStartTime=1522613264057, pause=100, retries=1}, java.io.IOException: Call to xxxxxxx/xxxxx:16202 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Cal l id=372918, waitTime=5002, operationTimeout=5000 expired. at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:157) at org.apache.hadoop.hbase.client.HTable.checkAndPut(HTable.java:1233) at org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:311) at org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:294) at org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:280) at org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:260) at org.apache.kylin.job.dao.ExecutableDao.writeJobOutputResource(ExecutableDao.java:104) at org.apache.kylin.job.dao.ExecutableDao.updateJobOutput(ExecutableDao.java:218) at org.apache.kylin.job.execution.ExecutableManager.addJobInfo(ExecutableManager.java:470) at org.apache.kylin.job.execution.ExecutableManager.addJobInfo(ExecutableManager.java:480) at org.apache.kylin.job.execution.AbstractExecutable.addExtraInfo(AbstractExecutable.java:352) at org.apache.kylin.job.execution.AbstractExecutable.setEndTime(AbstractExecutable.java:364) at org.apache.kylin.job.execution.AbstractExecutable.onExecuteFinished(AbstractExecutable.java:85) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:138) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:144) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Call to xxxxx/xxxx failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=372918, waitTime=5002, operationTimeout=5000 expired. at org.apache.hadoop.hbase.ipc.AbstractRpcClient.wrapException(AbstractRpcClient.java:292) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1271) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.mutate(ClientProtos.java:34082) at org.apache.hadoop.hbase.client.HTable$9.call(HTable.java:1226) at org.apache.hadoop.hbase.client.HTable$9.call(HTable.java:1216) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136) ... 19 more -- This message was sent by Atlassian JIRA (v7.6.3#76005)