Thank you for the updates. With Warm regards
Billy Liu 2018-03-27 16:37 GMT+08:00 凡梦星尘 <elkan1...@gmail.com>: > Hi guys: > > Sorry, It's my mistake. Because the Kylin check environment and said > without HCAT_HOME in path. So I had add it environment setting by manual. > But I add the hive2-hcat libs path then you were this error. Now I change > it down to hive-hcat libs that Kylin2.3 working well. > > Thanks. > > 2018-03-27 14:34 GMT+08:00 凡梦星尘 <elkan1...@gmail.com>: > >> At noon I had restore all settings in kylin.properties file, then cube >> build under MR mode without any problems. So there were very strange, I >> think the default setting for use same hadoop configure. Whats different >> about action copy the hive-site in custom hadoop conf directory. As well I >> had found deep about Hive source code, make sure the NoSuchMethodError were >> right. Because the constructor function is build in Hive2.1x version. Then >> try to upgrade the hive jars under spark jars folder, but still not >> working. So what can I do now? >> >> >> >> PS: >> >> 1. Use the Kylin2.0 spark guide line: https://kylin.apache. >> org/docs20/tutorial/cube_spark.html >> 2. Use beeline replace for CLI to connect Hive. >> 3. Upload the hive-site.xml file for your information. >> 4. Throws error source code of Kylin >> >> public void configureJob(Job job) { >> try { >> job.getConfiguration().addResource("hive-site.xml"); >> >> HCatInputFormat.setInput(job, dbName, tableName); >> job.setInputFormatClass(HCatInputFormat.class); >> } catch (IOException e) { >> throw new RuntimeException(e); >> } >> } >> >> >> >> Thanks. >> >> >> 2018-03-27 9:47 GMT+08:00 凡梦星尘 <elkan1...@gmail.com>: >> >>> Hi Billy: >>> >>> Thanks for your suggest, I will try it. Hope can response good news. >>> >>> 2018-03-27 9:11 GMT+08:00 Billy Liu <billy...@apache.org>: >>> >>>> For the first question, it seems some classpath conflict issue >>>> existing there "NoSuchMethodError", try google "HiveMetaStoreClient >>>> NoSuchMethodError" >>>> >>>> For the second question, to enable the dashboard, please follow the >>>> http://kylin.apache.org/docs23/howto/howto_setup_systemcube.html >>>> first. >>>> >>>> With Warm regards >>>> >>>> Billy Liu >>>> >>>> >>>> 2018-03-26 17:59 GMT+08:00 凡梦星尘 <elkan1...@gmail.com>: >>>> > Hi guys: >>>> > >>>> > Congratulation to Kylin release 2.3.0 with so many features. >>>> > >>>> > The last weekend I had try to upgrade this newest version to test ENV. >>>> All >>>> > things going is good and run well sample cube under MR mode. But when >>>> I try >>>> > to switch Spark mode not success. After fixed some problems that meet a >>>> > hard trouble not fixed. The error log see below: >>>> > >>>> > java.lang.RuntimeException: java.io.IOException: >>>> > com.google.common.util.concurrent.UncheckedExecutionException: >>>> > java.lang.RuntimeException: Unable to instantiate >>>> > org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv >>>> eMetaStoreClient >>>> > at org.apache.kylin.source.hive.H >>>> iveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:116) >>>> > at org.apache.kylin.engine.mr.ste >>>> ps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:121) >>>> > at org.apache.kylin.engine.mr.ste >>>> ps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:99) >>>> > at org.apache.kylin.engine.mr.com >>>> mon.MapReduceExecutable.doWork(MapReduceExecutable.java:130) >>>> > at org.apache.kylin.job.execution >>>> .AbstractExecutable.execute(AbstractExecutable.java:162) >>>> > at org.apache.kylin.job.execution >>>> .DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67) >>>> > at org.apache.kylin.job.execution >>>> .AbstractExecutable.execute(AbstractExecutable.java:162) >>>> > at org.apache.kylin.job.impl.thre >>>> adpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300) >>>> > at java.util.concurrent.ThreadPoo >>>> lExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>> > at java.util.concurrent.ThreadPoo >>>> lExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>> > at java.lang.Thread.run(Thread.java:748) >>>> > Caused by: java.io.IOException: >>>> > com.google.common.util.concurrent.UncheckedExecutionException: >>>> > java.lang.RuntimeException: Unable to instantiate >>>> > org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv >>>> eMetaStoreClient >>>> > at org.apache.hive.hcatalog.mapre >>>> duce.HCatInputFormat.setInput(HCatInputFormat.java:97) >>>> > at org.apache.hive.hcatalog.mapre >>>> duce.HCatInputFormat.setInput(HCatInputFormat.java:51) >>>> > at org.apache.kylin.source.hive.H >>>> iveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:113) >>>> > ... 10 more >>>> > Caused by: com.google.common.util.concurr >>>> ent.UncheckedExecutionException: >>>> > java.lang.RuntimeException: Unable to instantiate >>>> > org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv >>>> eMetaStoreClient >>>> > at com.google.common.cache.LocalC >>>> ache$Segment.get(LocalCache.java:2256) >>>> > at com.google.common.cache.LocalC >>>> ache.get(LocalCache.java:3985) >>>> > at com.google.common.cache.LocalC >>>> ache$LocalManualCache.get(LocalCache.java:4788) >>>> > at org.apache.hive.hcatalog.commo >>>> n.HiveClientCache.getOrCreate(HiveClientCache.java:292) >>>> > at org.apache.hive.hcatalog.commo >>>> n.HiveClientCache.get(HiveClientCache.java:267) >>>> > at org.apache.hive.hcatalog.commo >>>> n.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558) >>>> > at org.apache.hive.hcatalog.mapre >>>> duce.InitializeInput.getInputJobInfo(InitializeInput.java:104) >>>> > at org.apache.hive.hcatalog.mapre >>>> duce.InitializeInput.setInput(InitializeInput.java:88) >>>> > at org.apache.hive.hcatalog.mapre >>>> duce.HCatInputFormat.setInput(HCatInputFormat.java:95) >>>> > ... 12 more >>>> > Caused by: java.lang.RuntimeException: Unable to instantiate >>>> > org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv >>>> eMetaStoreClient >>>> > at org.apache.hadoop.hive.metasto >>>> re.MetaStoreUtils.newInstance(MetaStoreUtils.java:1566) >>>> > at org.apache.hadoop.hive.metasto >>>> re.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) >>>> > at org.apache.hadoop.hive.metasto >>>> re.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) >>>> > at org.apache.hadoop.hive.metasto >>>> re.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:124) >>>> > at org.apache.hive.hcatalog.commo >>>> n.HiveClientCache$5.call(HiveClientCache.java:297) >>>> > at org.apache.hive.hcatalog.commo >>>> n.HiveClientCache$5.call(HiveClientCache.java:292) >>>> > at com.google.common.cache.LocalC >>>> ache$LocalManualCache$1.load(LocalCache.java:4791) >>>> > at com.google.common.cache.LocalC >>>> ache$LoadingValueReference.loadFuture(LocalCache.java:3584) >>>> > at com.google.common.cache.LocalC >>>> ache$Segment.loadSync(LocalCache.java:2372) >>>> > at com.google.common.cache.LocalC >>>> ache$Segment.lockedGetOrLoad(LocalCache.java:2335) >>>> > at com.google.common.cache.LocalC >>>> ache$Segment.get(LocalCache.java:2250) >>>> > ... 20 more >>>> > Caused by: java.lang.reflect.InvocationTargetException >>>> > at sun.reflect.NativeConstructorA >>>> ccessorImpl.newInstance0(Native Method) >>>> > at sun.reflect.NativeConstructorA >>>> ccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >>>> > at sun.reflect.DelegatingConstruc >>>> torAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>>> > at java.lang.reflect.Constructor. >>>> newInstance(Constructor.java:423) >>>> > at org.apache.hadoop.hive.metasto >>>> re.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) >>>> > ... 30 more >>>> > Caused by: java.lang.NoSuchMethodError: >>>> > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>( >>>> Lorg/apache/hadoop/hive/conf/HiveConf;Lorg/apache/hadoop/hiv >>>> e/metastore/HiveMetaHookLoader;Ljava/lang/Boolean;)V >>>> > at org.apache.hive.hcatalog.commo >>>> n.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveCl >>>> ientCache.java:409) >>>> > ... 35 more >>>> > >>>> > result code:2 >>>> > >>>> > >>>> > At first time I had research some likely issues on forum but there not >>>> same >>>> > with me. Also try used the resolve ways but still not work. So someone >>>> can >>>> > give me more useful suggest. >>>> > >>>> > Thanks. >>>> > >>>> > >>>> > PS: >>>> > >>>> > 1. Hadoop ENV screenshot photo: >>>> > >>>> > http://77l54p.com1.z0.glb.clouddn.com/hdp_stack_versions_2.6 >>>> .4.png-alias >>>> > >>>> > 2. Other errors about dashboard page action, detail info see below >>>> logs: >>>> > >>>> > 2018-03-26 17:44:32,580 INFO [Query >>>> > 89e44f1b-5636-4c4b-98cb-423237effe81-49] service.QueryService:428 : >>>> > Using project: KYLIN_SYSTEM >>>> > 2018-03-26 17:44:32,585 INFO [Query >>>> > 89e44f1b-5636-4c4b-98cb-423237effe81-49] service.QueryService:429 : >>>> > The original query: select >>>> > count(*),sum(QUERY_TIME_COST)/(count(QUERY_TIME_COST)),max(Q >>>> UERY_TIME_COST),min(QUERY_TIME_COST) >>>> > from KYLIN.HIVE_METRICS_QUERY_QA where KDAY_DATE >= '2018-03-19' and >>>> > KDAY_DATE <= '2018-03-25' and PROJECT ='LEARN_KYLIN' and EXCEPTION = >>>> > 'NULL' >>>> > 2018-03-26 17:44:32,583 ERROR [http-bio-7070-exec-5] >>>> > controller.BasicController:61 : >>>> > java.lang.NullPointerException >>>> > at org.apache.kylin.rest.util.Que >>>> ryRequestLimits.<init>(QueryRequestLimits.java:107) >>>> > at org.apache.kylin.rest.service. >>>> QueryService.doQueryWithCache(QueryService.java:454) >>>> > at org.apache.kylin.rest.service. >>>> QueryService.doQueryWithCache(QueryService.java:390) >>>> > at org.apache.kylin.rest.controll >>>> er.DashboardController.getChartData(DashboardController.java:111) >>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> > at sun.reflect.NativeMethodAccess >>>> orImpl.invoke(NativeMethodAccessorImpl.java:62) >>>> > at sun.reflect.DelegatingMethodAc >>>> cessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> > at java.lang.reflect.Method.invoke(Method.java:498) >>>> > at org.springframework.web.method >>>> .support.InvocableHandlerMethod.doInvoke(InvocableHandlerMet >>>> hod.java:205) >>>> > at org.springframework.web.method >>>> .support.InvocableHandlerMethod.invokeForRequest(InvocableHa >>>> ndlerMethod.java:133) >>>> > at org.springframework.web.servle >>>> t.mvc.method.annotation.ServletInvocableHandlerMethod.invoke >>>> AndHandle(ServletInvocableHandlerMethod.java:97) >>>> > at org.springframework.web.servle >>>> t.mvc.method.annotation.RequestMappingHandlerAdapter.invokeH >>>> andlerMethod(RequestMappingHandlerAdapter.java:827) >>>> > at org.springframework.web.servle >>>> t.mvc.method.annotation.RequestMappingHandlerAdapter.handleI >>>> nternal(RequestMappingHandlerAdapter.java:738) >>>> > at org.springframework.web.servle >>>> t.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHan >>>> dlerMethodAdapter.java:85) >>>> > at org.springframework.web.servle >>>> t.DispatcherServlet.doDispatch(DispatcherServlet.java:967) >>>> > at org.springframework.web.servle >>>> t.DispatcherServlet.doService(DispatcherServlet.java:901) >>>> > at org.springframework.web.servle >>>> t.DispatcherServlet.doDispatch(DispatcherServlet.java:967) >>>> > at org.springframework.web.servle >>>> t.DispatcherServlet.doService(DispatcherServlet.java:901) >>>> > at org.springframework.web.servle >>>> t.FrameworkServlet.processRequest(FrameworkServlet.java:970) >>>> > at org.springframework.web.servle >>>> t.FrameworkServlet.doGet(FrameworkServlet.java:861) >>>> > at javax.servlet.http.HttpServlet >>>> .service(HttpServlet.java:624) >>>> > at org.springframework.web.servle >>>> t.FrameworkServlet.service(FrameworkServlet.java:846) >>>> > at javax.servlet.http.HttpServlet >>>> .service(HttpServlet.java:731) >>>> > at org.apache.catalina.core.Appli >>>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) >>>> > at org.apache.catalina.core.Appli >>>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) >>>> > at org.apache.tomcat.websocket.se >>>> rver.WsFilter.doFilter(WsFilter.java:52) >>>> > at org.apache.catalina.core.Appli >>>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) >>>> > at org.apache.catalina.core.Appli >>>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:317) >>>> > at org.springframework.security.w >>>> eb.access.intercept.FilterSecurityInterceptor.invoke(FilterS >>>> ecurityInterceptor.java:127) >>>> > at org.springframework.security.w >>>> eb.access.intercept.FilterSecurityInterceptor.doFilter(Filte >>>> rSecurityInterceptor.java:91) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.access.ExceptionTranslationFilter.doFilter(ExceptionTrans >>>> lationFilter.java:114) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.session.SessionManagementFilter.doFilter(SessionManagemen >>>> tFilter.java:137) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.authentication.AnonymousAuthenticationFilter.doFilter(Ano >>>> nymousAuthenticationFilter.java:111) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.servletapi.SecurityContextHolderAwareRequestFilter.doFilt >>>> er(SecurityContextHolderAwareRequestFilter.java:170) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.savedrequest.RequestCacheAwareFilter.doFilter(RequestCach >>>> eAwareFilter.java:63) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.authentication.www.BasicAuthenticationFilter.doFilterInte >>>> rnal(BasicAuthenticationFilter.java:158) >>>> > at org.springframework.web.filter >>>> .OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.authentication.AbstractAuthenticationProcessingFilter.doF >>>> ilter(AbstractAuthenticationProcessingFilter.java:200) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFi >>>> lter.java:64) >>>> > >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFi >>>> lter.java:64) >>>> > at org.springframework.web.filter >>>> .OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.context.request.async.WebAsyncManagerIntegrationFilter.do >>>> FilterInternal(WebAsyncManagerIntegrationFilter.java:56) >>>> > at org.springframework.web.filter >>>> .OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.context.SecurityContextPersistenceFilter.doFilter(Securit >>>> yContextPersistenceFilter.java:105) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>>> roxy.java:331) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) >>>> > at org.springframework.security.w >>>> eb.FilterChainProxy.doFilter(FilterChainProxy.java:177) >>>> > at org.springframework.web.filter >>>> .DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) >>>> > at org.springframework.web.filter >>>> .DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) >>>> > at org.apache.catalina.core.Appli >>>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) >>>> > at org.apache.catalina.core.Appli >>>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) >>>> > at com.thetransactioncompany.cors >>>> .CORSFilter.doFilter(CORSFilter.java:209) >>>> > at com.thetransactioncompany.cors >>>> .CORSFilter.doFilter(CORSFilter.java:244) >>>> > at org.apache.catalina.core.Appli >>>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) >>>> > at org.apache.catalina.core.Appli >>>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) >>>> > at org.apache.catalina.core.Stand >>>> ardWrapperValve.invoke(StandardWrapperValve.java:219) >>>> > at org.apache.catalina.core.Stand >>>> ardContextValve.invoke(StandardContextValve.java:110) >>>> > at org.apache.catalina.core.Stand >>>> ardHostValve.invoke(StandardHostValve.java:169) >>>> > at org.apache.catalina.valves.Err >>>> orReportValve.invoke(ErrorReportValve.java:103) >>>> > at org.apache.catalina.valves.Acc >>>> essLogValve.invoke(AccessLogValve.java:962) >>>> > at org.apache.catalina.core.Stand >>>> ardEngineValve.invoke(StandardEngineValve.java:116) >>>> > at org.apache.catalina.connector. >>>> CoyoteAdapter.service(CoyoteAdapter.java:445) >>>> > at org.apache.coyote.http11.Abstr >>>> actHttp11Processor.process(AbstractHttp11Processor.java:1115) >>>> > at org.apache.coyote.AbstractProt >>>> ocol$AbstractConnectionHandler.process(AbstractProtocol.java:637) >>>> > at org.apache.tomcat.util.net.JIo >>>> Endpoint$SocketProcessor.run(JIoEndpoint.java:316) >>>> > at java.util.concurrent.ThreadPoo >>>> lExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>> > at java.util.concurrent.ThreadPoo >>>> lExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>> > at org.apache.tomcat.util.threads >>>> .TaskThread$WrappingRunnable.run(TaskThread.java:61) >>>> > at java.lang.Thread.run(Thread.java:748) >>>> >>> >>> >>
