Hi guys: Sorry, It's my mistake. Because the Kylin check environment and said without HCAT_HOME in path. So I had add it environment setting by manual. But I add the hive2-hcat libs path then you were this error. Now I change it down to hive-hcat libs that Kylin2.3 working well.
Thanks. 2018-03-27 14:34 GMT+08:00 凡梦星尘 <elkan1...@gmail.com>: > At noon I had restore all settings in kylin.properties file, then cube > build under MR mode without any problems. So there were very strange, I > think the default setting for use same hadoop configure. Whats different > about action copy the hive-site in custom hadoop conf directory. As well I > had found deep about Hive source code, make sure the NoSuchMethodError were > right. Because the constructor function is build in Hive2.1x version. Then > try to upgrade the hive jars under spark jars folder, but still not > working. So what can I do now? > > > > PS: > > 1. Use the Kylin2.0 spark guide line: https://kylin.apache. > org/docs20/tutorial/cube_spark.html > 2. Use beeline replace for CLI to connect Hive. > 3. Upload the hive-site.xml file for your information. > 4. Throws error source code of Kylin > > public void configureJob(Job job) { > try { > job.getConfiguration().addResource("hive-site.xml"); > > HCatInputFormat.setInput(job, dbName, tableName); > job.setInputFormatClass(HCatInputFormat.class); > } catch (IOException e) { > throw new RuntimeException(e); > } > } > > > > Thanks. > > > 2018-03-27 9:47 GMT+08:00 凡梦星尘 <elkan1...@gmail.com>: > >> Hi Billy: >> >> Thanks for your suggest, I will try it. Hope can response good news. >> >> 2018-03-27 9:11 GMT+08:00 Billy Liu <billy...@apache.org>: >> >>> For the first question, it seems some classpath conflict issue >>> existing there "NoSuchMethodError", try google "HiveMetaStoreClient >>> NoSuchMethodError" >>> >>> For the second question, to enable the dashboard, please follow the >>> http://kylin.apache.org/docs23/howto/howto_setup_systemcube.html >>> first. >>> >>> With Warm regards >>> >>> Billy Liu >>> >>> >>> 2018-03-26 17:59 GMT+08:00 凡梦星尘 <elkan1...@gmail.com>: >>> > Hi guys: >>> > >>> > Congratulation to Kylin release 2.3.0 with so many features. >>> > >>> > The last weekend I had try to upgrade this newest version to test ENV. >>> All >>> > things going is good and run well sample cube under MR mode. But when >>> I try >>> > to switch Spark mode not success. After fixed some problems that meet a >>> > hard trouble not fixed. The error log see below: >>> > >>> > java.lang.RuntimeException: java.io.IOException: >>> > com.google.common.util.concurrent.UncheckedExecutionException: >>> > java.lang.RuntimeException: Unable to instantiate >>> > org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv >>> eMetaStoreClient >>> > at org.apache.kylin.source.hive.H >>> iveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:116) >>> > at org.apache.kylin.engine.mr.ste >>> ps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:121) >>> > at org.apache.kylin.engine.mr.ste >>> ps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:99) >>> > at org.apache.kylin.engine.mr.com >>> mon.MapReduceExecutable.doWork(MapReduceExecutable.java:130) >>> > at org.apache.kylin.job.execution >>> .AbstractExecutable.execute(AbstractExecutable.java:162) >>> > at org.apache.kylin.job.execution >>> .DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67) >>> > at org.apache.kylin.job.execution >>> .AbstractExecutable.execute(AbstractExecutable.java:162) >>> > at org.apache.kylin.job.impl.thre >>> adpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300) >>> > at java.util.concurrent.ThreadPoo >>> lExecutor.runWorker(ThreadPoolExecutor.java:1149) >>> > at java.util.concurrent.ThreadPoo >>> lExecutor$Worker.run(ThreadPoolExecutor.java:624) >>> > at java.lang.Thread.run(Thread.java:748) >>> > Caused by: java.io.IOException: >>> > com.google.common.util.concurrent.UncheckedExecutionException: >>> > java.lang.RuntimeException: Unable to instantiate >>> > org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv >>> eMetaStoreClient >>> > at org.apache.hive.hcatalog.mapre >>> duce.HCatInputFormat.setInput(HCatInputFormat.java:97) >>> > at org.apache.hive.hcatalog.mapre >>> duce.HCatInputFormat.setInput(HCatInputFormat.java:51) >>> > at org.apache.kylin.source.hive.H >>> iveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:113) >>> > ... 10 more >>> > Caused by: com.google.common.util.concurr >>> ent.UncheckedExecutionException: >>> > java.lang.RuntimeException: Unable to instantiate >>> > org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv >>> eMetaStoreClient >>> > at com.google.common.cache.LocalC >>> ache$Segment.get(LocalCache.java:2256) >>> > at com.google.common.cache.LocalC >>> ache.get(LocalCache.java:3985) >>> > at com.google.common.cache.LocalC >>> ache$LocalManualCache.get(LocalCache.java:4788) >>> > at org.apache.hive.hcatalog.commo >>> n.HiveClientCache.getOrCreate(HiveClientCache.java:292) >>> > at org.apache.hive.hcatalog.commo >>> n.HiveClientCache.get(HiveClientCache.java:267) >>> > at org.apache.hive.hcatalog.commo >>> n.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558) >>> > at org.apache.hive.hcatalog.mapre >>> duce.InitializeInput.getInputJobInfo(InitializeInput.java:104) >>> > at org.apache.hive.hcatalog.mapre >>> duce.InitializeInput.setInput(InitializeInput.java:88) >>> > at org.apache.hive.hcatalog.mapre >>> duce.HCatInputFormat.setInput(HCatInputFormat.java:95) >>> > ... 12 more >>> > Caused by: java.lang.RuntimeException: Unable to instantiate >>> > org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv >>> eMetaStoreClient >>> > at org.apache.hadoop.hive.metasto >>> re.MetaStoreUtils.newInstance(MetaStoreUtils.java:1566) >>> > at org.apache.hadoop.hive.metasto >>> re.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) >>> > at org.apache.hadoop.hive.metasto >>> re.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) >>> > at org.apache.hadoop.hive.metasto >>> re.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:124) >>> > at org.apache.hive.hcatalog.commo >>> n.HiveClientCache$5.call(HiveClientCache.java:297) >>> > at org.apache.hive.hcatalog.commo >>> n.HiveClientCache$5.call(HiveClientCache.java:292) >>> > at com.google.common.cache.LocalC >>> ache$LocalManualCache$1.load(LocalCache.java:4791) >>> > at com.google.common.cache.LocalC >>> ache$LoadingValueReference.loadFuture(LocalCache.java:3584) >>> > at com.google.common.cache.LocalC >>> ache$Segment.loadSync(LocalCache.java:2372) >>> > at com.google.common.cache.LocalC >>> ache$Segment.lockedGetOrLoad(LocalCache.java:2335) >>> > at com.google.common.cache.LocalC >>> ache$Segment.get(LocalCache.java:2250) >>> > ... 20 more >>> > Caused by: java.lang.reflect.InvocationTargetException >>> > at sun.reflect.NativeConstructorA >>> ccessorImpl.newInstance0(Native Method) >>> > at sun.reflect.NativeConstructorA >>> ccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >>> > at sun.reflect.DelegatingConstruc >>> torAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>> > at java.lang.reflect.Constructor. >>> newInstance(Constructor.java:423) >>> > at org.apache.hadoop.hive.metasto >>> re.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) >>> > ... 30 more >>> > Caused by: java.lang.NoSuchMethodError: >>> > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>( >>> Lorg/apache/hadoop/hive/conf/HiveConf;Lorg/apache/hadoop/hiv >>> e/metastore/HiveMetaHookLoader;Ljava/lang/Boolean;)V >>> > at org.apache.hive.hcatalog.commo >>> n.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveCl >>> ientCache.java:409) >>> > ... 35 more >>> > >>> > result code:2 >>> > >>> > >>> > At first time I had research some likely issues on forum but there not >>> same >>> > with me. Also try used the resolve ways but still not work. So someone >>> can >>> > give me more useful suggest. >>> > >>> > Thanks. >>> > >>> > >>> > PS: >>> > >>> > 1. Hadoop ENV screenshot photo: >>> > >>> > http://77l54p.com1.z0.glb.clouddn.com/hdp_stack_versions_2.6 >>> .4.png-alias >>> > >>> > 2. Other errors about dashboard page action, detail info see below >>> logs: >>> > >>> > 2018-03-26 17:44:32,580 INFO [Query >>> > 89e44f1b-5636-4c4b-98cb-423237effe81-49] service.QueryService:428 : >>> > Using project: KYLIN_SYSTEM >>> > 2018-03-26 17:44:32,585 INFO [Query >>> > 89e44f1b-5636-4c4b-98cb-423237effe81-49] service.QueryService:429 : >>> > The original query: select >>> > count(*),sum(QUERY_TIME_COST)/(count(QUERY_TIME_COST)),max(Q >>> UERY_TIME_COST),min(QUERY_TIME_COST) >>> > from KYLIN.HIVE_METRICS_QUERY_QA where KDAY_DATE >= '2018-03-19' and >>> > KDAY_DATE <= '2018-03-25' and PROJECT ='LEARN_KYLIN' and EXCEPTION = >>> > 'NULL' >>> > 2018-03-26 17:44:32,583 ERROR [http-bio-7070-exec-5] >>> > controller.BasicController:61 : >>> > java.lang.NullPointerException >>> > at org.apache.kylin.rest.util.Que >>> ryRequestLimits.<init>(QueryRequestLimits.java:107) >>> > at org.apache.kylin.rest.service. >>> QueryService.doQueryWithCache(QueryService.java:454) >>> > at org.apache.kylin.rest.service. >>> QueryService.doQueryWithCache(QueryService.java:390) >>> > at org.apache.kylin.rest.controll >>> er.DashboardController.getChartData(DashboardController.java:111) >>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> > at sun.reflect.NativeMethodAccess >>> orImpl.invoke(NativeMethodAccessorImpl.java:62) >>> > at sun.reflect.DelegatingMethodAc >>> cessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> > at java.lang.reflect.Method.invoke(Method.java:498) >>> > at org.springframework.web.method >>> .support.InvocableHandlerMethod.doInvoke(InvocableHandlerMet >>> hod.java:205) >>> > at org.springframework.web.method >>> .support.InvocableHandlerMethod.invokeForRequest(InvocableHa >>> ndlerMethod.java:133) >>> > at org.springframework.web.servle >>> t.mvc.method.annotation.ServletInvocableHandlerMethod.invoke >>> AndHandle(ServletInvocableHandlerMethod.java:97) >>> > at org.springframework.web.servle >>> t.mvc.method.annotation.RequestMappingHandlerAdapter.invokeH >>> andlerMethod(RequestMappingHandlerAdapter.java:827) >>> > at org.springframework.web.servle >>> t.mvc.method.annotation.RequestMappingHandlerAdapter.handleI >>> nternal(RequestMappingHandlerAdapter.java:738) >>> > at org.springframework.web.servle >>> t.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHan >>> dlerMethodAdapter.java:85) >>> > at org.springframework.web.servle >>> t.DispatcherServlet.doDispatch(DispatcherServlet.java:967) >>> > at org.springframework.web.servle >>> t.DispatcherServlet.doService(DispatcherServlet.java:901) >>> > at org.springframework.web.servle >>> t.DispatcherServlet.doDispatch(DispatcherServlet.java:967) >>> > at org.springframework.web.servle >>> t.DispatcherServlet.doService(DispatcherServlet.java:901) >>> > at org.springframework.web.servle >>> t.FrameworkServlet.processRequest(FrameworkServlet.java:970) >>> > at org.springframework.web.servle >>> t.FrameworkServlet.doGet(FrameworkServlet.java:861) >>> > at javax.servlet.http.HttpServlet >>> .service(HttpServlet.java:624) >>> > at org.springframework.web.servle >>> t.FrameworkServlet.service(FrameworkServlet.java:846) >>> > at javax.servlet.http.HttpServlet >>> .service(HttpServlet.java:731) >>> > at org.apache.catalina.core.Appli >>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) >>> > at org.apache.catalina.core.Appli >>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) >>> > at org.apache.tomcat.websocket.se >>> rver.WsFilter.doFilter(WsFilter.java:52) >>> > at org.apache.catalina.core.Appli >>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) >>> > at org.apache.catalina.core.Appli >>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:317) >>> > at org.springframework.security.w >>> eb.access.intercept.FilterSecurityInterceptor.invoke(FilterS >>> ecurityInterceptor.java:127) >>> > at org.springframework.security.w >>> eb.access.intercept.FilterSecurityInterceptor.doFilter(Filte >>> rSecurityInterceptor.java:91) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.access.ExceptionTranslationFilter.doFilter(ExceptionTrans >>> lationFilter.java:114) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.session.SessionManagementFilter.doFilter(SessionManagemen >>> tFilter.java:137) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.authentication.AnonymousAuthenticationFilter.doFilter(Ano >>> nymousAuthenticationFilter.java:111) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.servletapi.SecurityContextHolderAwareRequestFilter.doFilt >>> er(SecurityContextHolderAwareRequestFilter.java:170) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.savedrequest.RequestCacheAwareFilter.doFilter(RequestCach >>> eAwareFilter.java:63) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.authentication.www.BasicAuthenticationFilter.doFilterInte >>> rnal(BasicAuthenticationFilter.java:158) >>> > at org.springframework.web.filter >>> .OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.authentication.AbstractAuthenticationProcessingFilter.doF >>> ilter(AbstractAuthenticationProcessingFilter.java:200) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFi >>> lter.java:64) >>> > >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFi >>> lter.java:64) >>> > at org.springframework.web.filter >>> .OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.context.request.async.WebAsyncManagerIntegrationFilter.do >>> FilterInternal(WebAsyncManagerIntegrationFilter.java:56) >>> > at org.springframework.web.filter >>> .OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.context.SecurityContextPersistenceFilter.doFilter(Securit >>> yContextPersistenceFilter.java:105) >>> > at org.springframework.security.w >>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP >>> roxy.java:331) >>> > at org.springframework.security.w >>> eb.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) >>> > at org.springframework.security.w >>> eb.FilterChainProxy.doFilter(FilterChainProxy.java:177) >>> > at org.springframework.web.filter >>> .DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) >>> > at org.springframework.web.filter >>> .DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) >>> > at org.apache.catalina.core.Appli >>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) >>> > at org.apache.catalina.core.Appli >>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) >>> > at com.thetransactioncompany.cors >>> .CORSFilter.doFilter(CORSFilter.java:209) >>> > at com.thetransactioncompany.cors >>> .CORSFilter.doFilter(CORSFilter.java:244) >>> > at org.apache.catalina.core.Appli >>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) >>> > at org.apache.catalina.core.Appli >>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) >>> > at org.apache.catalina.core.Stand >>> ardWrapperValve.invoke(StandardWrapperValve.java:219) >>> > at org.apache.catalina.core.Stand >>> ardContextValve.invoke(StandardContextValve.java:110) >>> > at org.apache.catalina.core.Stand >>> ardHostValve.invoke(StandardHostValve.java:169) >>> > at org.apache.catalina.valves.Err >>> orReportValve.invoke(ErrorReportValve.java:103) >>> > at org.apache.catalina.valves.Acc >>> essLogValve.invoke(AccessLogValve.java:962) >>> > at org.apache.catalina.core.Stand >>> ardEngineValve.invoke(StandardEngineValve.java:116) >>> > at org.apache.catalina.connector. >>> CoyoteAdapter.service(CoyoteAdapter.java:445) >>> > at org.apache.coyote.http11.Abstr >>> actHttp11Processor.process(AbstractHttp11Processor.java:1115) >>> > at org.apache.coyote.AbstractProt >>> ocol$AbstractConnectionHandler.process(AbstractProtocol.java:637) >>> > at org.apache.tomcat.util.net.JIo >>> Endpoint$SocketProcessor.run(JIoEndpoint.java:316) >>> > at java.util.concurrent.ThreadPoo >>> lExecutor.runWorker(ThreadPoolExecutor.java:1149) >>> > at java.util.concurrent.ThreadPoo >>> lExecutor$Worker.run(ThreadPoolExecutor.java:624) >>> > at org.apache.tomcat.util.threads >>> .TaskThread$WrappingRunnable.run(TaskThread.java:61) >>> > at java.lang.Thread.run(Thread.java:748) >>> >> >> >
