[ https://issues.apache.org/jira/browse/KYLIN-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15689317#comment-15689317 ]
Billy(Yiming) Liu commented on KYLIN-2225: ------------------------------------------ It only happens for "Top N" query, or for each query? >From the log, I will suggest checking HBase healthy first. > "Error in coprocessor" problem while executing TOP N query > ---------------------------------------------------------- > > Key: KYLIN-2225 > URL: https://issues.apache.org/jira/browse/KYLIN-2225 > Project: Kylin > Issue Type: Bug > Components: Storage - HBase > Affects Versions: v1.5.4.1 > Reporter: Yangliang Li > Assignee: liyang > > Software information: > * Kylin: apache-kylin-1.5.4.1-cdh5.7 > * Hadoop: CHD 5.7 > * HBase: 1.2.0-cdh5.7.1 > * HBase coprocessor: kylin-coprocessor-1.5.4.1-0.jar > I successfully built a cube, but have problem while executing a TOP N query. > Cube > Here is the SQL statement: > ``` > select spu_id, sum(sales) from daily_deal_wide_view group by spu_id order by > sum(sales) desc limit 100 > ``` > Error message: > ``` > Error while executing SQL "select spu_id, sum(sales) from > daily_deal_wide_view group by spu_id order by sum(sales) desc limit 100": > Error in coprocessor > ``` > Log details: > ``` > 2016-11-23 15:01:56,674 WARN [kylin-coproc--pool2-t16] > ipc.CoprocessorRpcChannel:58 : Call failed on IOException > java.net.SocketTimeoutException: callTimeout=60000, callDuration=69049: row > '' on table 'KYLIN_O8HMHDKRD0' at > region=KYLIN_O8HMHDKRD0,,1479880633595.015f60fd1695b68f59981e1ba3c7d5fd., > hostname=ym-094,60020,1479183824861, seqNum=11 > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95) > at > org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56) > at > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:4013) > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:179) > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:175) > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1797) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.io.IOException: Call to ym-094/172.16.24.94:60020 failed on > local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call > id=406, waitTime=60001, operationTimeout=60000 expired. > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.wrapException(AbstractRpcClient.java:291) > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1272) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:226) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:331) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:34118) > at > org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1620) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126) > ... 10 more > Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=406, > waitTime=60001, operationTimeout=60000 expired. > at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:73) > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1246) > ... 17 more > 2016-11-23 15:01:56,678 WARN [pool-9-thread-15] client.HTable:1811 : Error > calling coprocessor service > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService > for row \x00\x00 > java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: > callTimeout=60000, callDuration=69049: row '' on table 'KYLIN_O8HMHDKRD0' at > region=KYLIN_O8HMHDKRD0,,1479880633595.015f60fd1695b68f59981e1ba3c7d5fd., > hostname=ym-094,60020,1479183824861, seqNum=11 > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:188) > at > org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1809) > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1.run(CubeHBaseEndpointRPC.java:174) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.net.SocketTimeoutException: callTimeout=60000, > callDuration=69049: row '' on table 'KYLIN_O8HMHDKRD0' at > region=KYLIN_O8HMHDKRD0,,1479880633595.015f60fd1695b68f59981e1ba3c7d5fd., > hostname=ym-094,60020,1479183824861, seqNum=11 > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95) > at > org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56) > at > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:4013) > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:179) > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:175) > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1797) > ... 4 more > Caused by: java.io.IOException: Call to ym-094/172.16.24.94:60020 failed on > local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call > id=406, waitTime=60001, operationTimeout=60000 expired. > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.wrapException(AbstractRpcClient.java:291) > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1272) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:226) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:331) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:34118) > at > org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1620) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126) > ... 10 more > Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=406, > waitTime=60001, operationTimeout=60000 expired. > at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:73) > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1246) > ... 17 more > 2016-11-23 15:01:56,679 ERROR [pool-9-thread-15] v2.CubeHBaseEndpointRPC:212 > : <sub-thread for GTScanRequest 6979cfa6> Error when visiting cubes by > endpoint > java.net.SocketTimeoutException: callTimeout=60000, callDuration=69049: row > '' on table 'KYLIN_O8HMHDKRD0' at > region=KYLIN_O8HMHDKRD0,,1479880633595.015f60fd1695b68f59981e1ba3c7d5fd., > hostname=ym-094,60020,1479183824861, seqNum=11 > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95) > at > org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56) > at > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:4013) > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:179) > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:175) > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1797) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.io.IOException: Call to ym-094/172.16.24.94:60020 failed on > local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call > id=406, waitTime=60001, operationTimeout=60000 expired. > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.wrapException(AbstractRpcClient.java:291) > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1272) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:226) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:331) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:34118) > at > org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1620) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126) > ... 10 more > Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=406, > waitTime=60001, operationTimeout=60000 expired. > at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:73) > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1246) > ... 17 more > 2016-11-23 15:02:06,680 ERROR [http-bio-7070-exec-8] > controller.QueryController:208 : Exception when execute sql > java.sql.SQLException: Error while executing SQL "select spu_id, sum(sales) > from daily_deal_wide_view group by spu_id order by sum(sales) desc limit > 100": Error in coprocessor > at org.apache.calcite.avatica.Helper.createException(Helper.java:56) > at org.apache.calcite.avatica.Helper.createException(Helper.java:41) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:147) > at > org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:208) > at > org.apache.kylin.rest.service.QueryService.execute(QueryService.java:408) > at > org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:320) > at > org.apache.kylin.rest.service.QueryService.query(QueryService.java:127) > at > org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:191) > at > org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:743) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:672) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:82) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:933) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:867) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:951) > at > org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:853) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:650) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:827) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:201) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) > at > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209) > at > com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.RuntimeException: Error in coprocessor > at > org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:95) > at > org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:37) > at > com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) > at com.google.common.collect.Iterators$5.hasNext(Iterators.java:597) > at > org.apache.kylin.storage.gtrecord.SegmentCubeTupleIterator.hasNext(SegmentCubeTupleIterator.java:96) > at com.google.common.collect.Iterators$5.hasNext(Iterators.java:596) > at > org.apache.kylin.storage.gtrecord.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:84) > at > org.apache.kylin.query.enumerator.OLAPEnumerator.moveNext(OLAPEnumerator.java:67) > at Baz$1$1.moveNext(Unknown Source) > at > org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:823) > at > org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:758) > at > org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:302) > at Baz.bind(Unknown Source) > at > org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:327) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:282) > at > org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:553) > at > org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:544) > at > org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:187) > at > org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:65) > at > org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44) > at > org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:605) > at > org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:599) > at > org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:613) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:139) > ... 77 more > Caused by: java.net.SocketTimeoutException: callTimeout=60000, > callDuration=69049: row '' on table 'KYLIN_O8HMHDKRD0' at > region=KYLIN_O8HMHDKRD0,,1479880633595.015f60fd1695b68f59981e1ba3c7d5fd., > hostname=ym-094,60020,1479183824861, seqNum=11 > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95) > at > org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56) > at > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:4013) > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:179) > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:175) > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1797) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ... 1 more > Caused by: java.io.IOException: Call to ym-094/172.16.24.94:60020 failed on > local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call > id=406, waitTime=60001, operationTimeout=60000 expired. > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.wrapException(AbstractRpcClient.java:291) > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1272) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:226) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:331) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:34118) > at > org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1620) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92) > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126) > ... 10 more > Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=406, > waitTime=60001, operationTimeout=60000 expired. > at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:73) > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1246) > ... 17 more > 2016-11-23 15:02:06,686 INFO [http-bio-7070-exec-8] service.QueryService:259 > : > ==========================[QUERY]=============================== > SQL: select spu_id, sum(sales) from daily_deal_wide_view group by spu_id > order by sum(sales) desc limit 100 > User: ADMIN > Success: false > Duration: 0.0 > Project: tm_dress_data > Realization Names: [basic_dress_daily_deal_cube2] > Cuboid Ids: [1] > Total scan count: 0 > Result row count: 0 > Accept Partial: true > Is Partial Result: false > Hit Exception Cache: false > Storage cache used: false > Message: Error while executing SQL "select spu_id, sum(sales) from > daily_deal_wide_view group by spu_id order by sum(sales) desc limit 100": > Error in coprocessor > ==========================[QUERY]=============================== > 2016-11-23 15:02:06,690 ERROR [http-bio-7070-exec-8] > controller.BasicController:44 : > org.apache.kylin.rest.exception.InternalErrorException: Error while executing > SQL "select spu_id, sum(sales) from daily_deal_wide_view group by spu_id > order by sum(sales) desc limit 100": Error in coprocessor > at > org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:225) > at > org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:743) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:672) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:82) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:933) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:867) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:951) > at > org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:853) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:650) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:827) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:201) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) > at > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209) > at > com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > ``` -- This message was sent by Atlassian JIRA (v6.3.4#6332)