Hi zhenya： Why not sure? Now all things is going well.
2018-03-27 16:45 GMT+08:00 zhenya Sun <toke...@126.com>: > excuse me are you sure this is ready to use according to what you said? > > > > 在 2018年3月27日，下午4:37，凡梦星尘 <elkan1...@gmail.com> 写道： > > > > Hi guys: > > > > Sorry, It's my mistake. Because the Kylin check environment and said > > without HCAT_HOME in path. So I had add it environment setting by manual. > > But I add the hive2-hcat libs path then you were this error. Now I change > > it down to hive-hcat libs that Kylin2.3 working well. > > > > Thanks. > > > > 2018-03-27 14:34 GMT+08:00 凡梦星尘 <elkan1...@gmail.com>: > > > >> At noon I had restore all settings in kylin.properties file, then cube > >> build under MR mode without any problems. So there were very strange, I > >> think the default setting for use same hadoop configure. Whats different > >> about action copy the hive-site in custom hadoop conf directory. As > well I > >> had found deep about Hive source code, make sure the NoSuchMethodError > were > >> right. Because the constructor function is build in Hive2.1x version. > Then > >> try to upgrade the hive jars under spark jars folder, but still not > >> working. So what can I do now? > >> > >> > >> > >> PS: > >> > >> 1. Use the Kylin2.0 spark guide line: https://kylin.apache. > >> org/docs20/tutorial/cube_spark.html > >> 2. Use beeline replace for CLI to connect Hive. > >> 3. Upload the hive-site.xml file for your information. > >> 4. Throws error source code of Kylin > >> > >> public void configureJob(Job job) { > >> try { > >> job.getConfiguration().addResource("hive-site.xml"); > >> > >> HCatInputFormat.setInput(job, dbName, tableName); > >> job.setInputFormatClass(HCatInputFormat.class); > >> } catch (IOException e) { > >> throw new RuntimeException(e); > >> } > >> } > >> > >> > >> > >> Thanks. > >> > >> > >> 2018-03-27 9:47 GMT+08:00 凡梦星尘 <elkan1...@gmail.com>: > >> > >>> Hi Billy: > >>> > >>> Thanks for your suggest, I will try it. Hope can response good news. > >>> > >>> 2018-03-27 9:11 GMT+08:00 Billy Liu <billy...@apache.org>: > >>> > >>>> For the first question, it seems some classpath conflict issue > >>>> existing there "NoSuchMethodError", try google "HiveMetaStoreClient > >>>> NoSuchMethodError" > >>>> > >>>> For the second question, to enable the dashboard, please follow the > >>>> http://kylin.apache.org/docs23/howto/howto_setup_systemcube.html > >>>> first. > >>>> > >>>> With Warm regards > >>>> > >>>> Billy Liu > >>>> > >>>> > >>>> 2018-03-26 17:59 GMT+08:00 凡梦星尘 <elkan1...@gmail.com>: > >>>>> Hi guys: > >>>>> > >>>>> Congratulation to Kylin release 2.3.0 with so many features. > >>>>> > >>>>> The last weekend I had try to upgrade this newest version to test > ENV. > >>>> All > >>>>> things going is good and run well sample cube under MR mode. But when > >>>> I try > >>>>> to switch Spark mode not success. After fixed some problems that > meet a > >>>>> hard trouble not fixed. The error log see below: > >>>>> > >>>>> java.lang.RuntimeException: java.io.IOException: > >>>>> com.google.common.util.concurrent.UncheckedExecutionException: > >>>>> java.lang.RuntimeException: Unable to instantiate > >>>>> org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv > >>>> eMetaStoreClient > >>>>> at org.apache.kylin.source.hive.H > >>>> iveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:116) > >>>>> at org.apache.kylin.engine.mr.ste > >>>> ps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java: > 121) > >>>>> at org.apache.kylin.engine.mr.ste > >>>> ps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:99) > >>>>> at org.apache.kylin.engine.mr.com > >>>> mon.MapReduceExecutable.doWork(MapReduceExecutable.java:130) > >>>>> at org.apache.kylin.job.execution > >>>> .AbstractExecutable.execute(AbstractExecutable.java:162) > >>>>> at org.apache.kylin.job.execution > >>>> .DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67) > >>>>> at org.apache.kylin.job.execution > >>>> .AbstractExecutable.execute(AbstractExecutable.java:162) > >>>>> at org.apache.kylin.job.impl.thre > >>>> adpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300) > >>>>> at java.util.concurrent.ThreadPoo > >>>> lExecutor.runWorker(ThreadPoolExecutor.java:1149) > >>>>> at java.util.concurrent.ThreadPoo > >>>> lExecutor$Worker.run(ThreadPoolExecutor.java:624) > >>>>> at java.lang.Thread.run(Thread.java:748) > >>>>> Caused by: java.io.IOException: > >>>>> com.google.common.util.concurrent.UncheckedExecutionException: > >>>>> java.lang.RuntimeException: Unable to instantiate > >>>>> org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv > >>>> eMetaStoreClient > >>>>> at org.apache.hive.hcatalog.mapre > >>>> duce.HCatInputFormat.setInput(HCatInputFormat.java:97) > >>>>> at org.apache.hive.hcatalog.mapre > >>>> duce.HCatInputFormat.setInput(HCatInputFormat.java:51) > >>>>> at org.apache.kylin.source.hive.H > >>>> iveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:113) > >>>>> ... 10 more > >>>>> Caused by: com.google.common.util.concurr > >>>> ent.UncheckedExecutionException: > >>>>> java.lang.RuntimeException: Unable to instantiate > >>>>> org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv > >>>> eMetaStoreClient > >>>>> at com.google.common.cache.LocalC > >>>> ache$Segment.get(LocalCache.java:2256) > >>>>> at com.google.common.cache.LocalC > >>>> ache.get(LocalCache.java:3985) > >>>>> at com.google.common.cache.LocalC > >>>> ache$LocalManualCache.get(LocalCache.java:4788) > >>>>> at org.apache.hive.hcatalog.commo > >>>> n.HiveClientCache.getOrCreate(HiveClientCache.java:292) > >>>>> at org.apache.hive.hcatalog.commo > >>>> n.HiveClientCache.get(HiveClientCache.java:267) > >>>>> at org.apache.hive.hcatalog.commo > >>>> n.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558) > >>>>> at org.apache.hive.hcatalog.mapre > >>>> duce.InitializeInput.getInputJobInfo(InitializeInput.java:104) > >>>>> at org.apache.hive.hcatalog.mapre > >>>> duce.InitializeInput.setInput(InitializeInput.java:88) > >>>>> at org.apache.hive.hcatalog.mapre > >>>> duce.HCatInputFormat.setInput(HCatInputFormat.java:95) > >>>>> ... 12 more > >>>>> Caused by: java.lang.RuntimeException: Unable to instantiate > >>>>> org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiv > >>>> eMetaStoreClient > >>>>> at org.apache.hadoop.hive.metasto > >>>> re.MetaStoreUtils.newInstance(MetaStoreUtils.java:1566) > >>>>> at org.apache.hadoop.hive.metasto > >>>> re.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) > >>>>> at org.apache.hadoop.hive.metasto > >>>> re.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) > >>>>> at org.apache.hadoop.hive.metasto > >>>> re.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:124) > >>>>> at org.apache.hive.hcatalog.commo > >>>> n.HiveClientCache$5.call(HiveClientCache.java:297) > >>>>> at org.apache.hive.hcatalog.commo > >>>> n.HiveClientCache$5.call(HiveClientCache.java:292) > >>>>> at com.google.common.cache.LocalC > >>>> ache$LocalManualCache$1.load(LocalCache.java:4791) > >>>>> at com.google.common.cache.LocalC > >>>> ache$LoadingValueReference.loadFuture(LocalCache.java:3584) > >>>>> at com.google.common.cache.LocalC > >>>> ache$Segment.loadSync(LocalCache.java:2372) > >>>>> at com.google.common.cache.LocalC > >>>> ache$Segment.lockedGetOrLoad(LocalCache.java:2335) > >>>>> at com.google.common.cache.LocalC > >>>> ache$Segment.get(LocalCache.java:2250) > >>>>> ... 20 more > >>>>> Caused by: java.lang.reflect.InvocationTargetException > >>>>> at sun.reflect.NativeConstructorA > >>>> ccessorImpl.newInstance0(Native Method) > >>>>> at sun.reflect.NativeConstructorA > >>>> ccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > >>>>> at sun.reflect.DelegatingConstruc > >>>> torAccessorImpl.newInstance(DelegatingConstructorAccessorI > mpl.java:45) > >>>>> at java.lang.reflect.Constructor. > >>>> newInstance(Constructor.java:423) > >>>>> at org.apache.hadoop.hive.metasto > >>>> re.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) > >>>>> ... 30 more > >>>>> Caused by: java.lang.NoSuchMethodError: > >>>>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>( > >>>> Lorg/apache/hadoop/hive/conf/HiveConf;Lorg/apache/hadoop/hiv > >>>> e/metastore/HiveMetaHookLoader;Ljava/lang/Boolean;)V > >>>>> at org.apache.hive.hcatalog.commo > >>>> n.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveCl > >>>> ientCache.java:409) > >>>>> ... 35 more > >>>>> > >>>>> result code:2 > >>>>> > >>>>> > >>>>> At first time I had research some likely issues on forum but there > not > >>>> same > >>>>> with me. Also try used the resolve ways but still not work. So > someone > >>>> can > >>>>> give me more useful suggest. > >>>>> > >>>>> Thanks. > >>>>> > >>>>> > >>>>> PS: > >>>>> > >>>>> 1. Hadoop ENV screenshot photo: > >>>>> > >>>>> http://77l54p.com1.z0.glb.clouddn.com/hdp_stack_versions_2.6 > >>>> .4.png-alias > >>>>> > >>>>> 2. Other errors about dashboard page action, detail info see below > >>>> logs: > >>>>> > >>>>> 2018-03-26 17:44:32,580 INFO [Query > >>>>> 89e44f1b-5636-4c4b-98cb-423237effe81-49] service.QueryService:428 : > >>>>> Using project: KYLIN_SYSTEM > >>>>> 2018-03-26 17:44:32,585 INFO [Query > >>>>> 89e44f1b-5636-4c4b-98cb-423237effe81-49] service.QueryService:429 : > >>>>> The original query: select > >>>>> count(*),sum(QUERY_TIME_COST)/(count(QUERY_TIME_COST)),max(Q > >>>> UERY_TIME_COST),min(QUERY_TIME_COST) > >>>>> from KYLIN.HIVE_METRICS_QUERY_QA where KDAY_DATE >= '2018-03-19' and > >>>>> KDAY_DATE <= '2018-03-25' and PROJECT ='LEARN_KYLIN' and EXCEPTION = > >>>>> 'NULL' > >>>>> 2018-03-26 17:44:32,583 ERROR [http-bio-7070-exec-5] > >>>>> controller.BasicController:61 : > >>>>> java.lang.NullPointerException > >>>>> at org.apache.kylin.rest.util.Que > >>>> ryRequestLimits.<init>(QueryRequestLimits.java:107) > >>>>> at org.apache.kylin.rest.service. > >>>> QueryService.doQueryWithCache(QueryService.java:454) > >>>>> at org.apache.kylin.rest.service. > >>>> QueryService.doQueryWithCache(QueryService.java:390) > >>>>> at org.apache.kylin.rest.controll > >>>> er.DashboardController.getChartData(DashboardController.java:111) > >>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > >>>>> at sun.reflect.NativeMethodAccess > >>>> orImpl.invoke(NativeMethodAccessorImpl.java:62) > >>>>> at sun.reflect.DelegatingMethodAc > >>>> cessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>>>> at java.lang.reflect.Method.invoke(Method.java:498) > >>>>> at org.springframework.web.method > >>>> .support.InvocableHandlerMethod.doInvoke(InvocableHandlerMet > >>>> hod.java:205) > >>>>> at org.springframework.web.method > >>>> .support.InvocableHandlerMethod.invokeForRequest(InvocableHa > >>>> ndlerMethod.java:133) > >>>>> at org.springframework.web.servle > >>>> t.mvc.method.annotation.ServletInvocableHandlerMethod.invoke > >>>> AndHandle(ServletInvocableHandlerMethod.java:97) > >>>>> at org.springframework.web.servle > >>>> t.mvc.method.annotation.RequestMappingHandlerAdapter.invokeH > >>>> andlerMethod(RequestMappingHandlerAdapter.java:827) > >>>>> at org.springframework.web.servle > >>>> t.mvc.method.annotation.RequestMappingHandlerAdapter.handleI > >>>> nternal(RequestMappingHandlerAdapter.java:738) > >>>>> at org.springframework.web.servle > >>>> t.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHan > >>>> dlerMethodAdapter.java:85) > >>>>> at org.springframework.web.servle > >>>> t.DispatcherServlet.doDispatch(DispatcherServlet.java:967) > >>>>> at org.springframework.web.servle > >>>> t.DispatcherServlet.doService(DispatcherServlet.java:901) > >>>>> at org.springframework.web.servle > >>>> t.DispatcherServlet.doDispatch(DispatcherServlet.java:967) > >>>>> at org.springframework.web.servle > >>>> t.DispatcherServlet.doService(DispatcherServlet.java:901) > >>>>> at org.springframework.web.servle > >>>> t.FrameworkServlet.processRequest(FrameworkServlet.java:970) > >>>>> at org.springframework.web.servle > >>>> t.FrameworkServlet.doGet(FrameworkServlet.java:861) > >>>>> at javax.servlet.http.HttpServlet > >>>> .service(HttpServlet.java:624) > >>>>> at org.springframework.web.servle > >>>> t.FrameworkServlet.service(FrameworkServlet.java:846) > >>>>> at javax.servlet.http.HttpServlet > >>>> .service(HttpServlet.java:731) > >>>>> at org.apache.catalina.core.Appli > >>>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > >>>>> at org.apache.catalina.core.Appli > >>>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) > >>>>> at org.apache.tomcat.websocket.se > >>>> rver.WsFilter.doFilter(WsFilter.java:52) > >>>>> at org.apache.catalina.core.Appli > >>>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > >>>>> at org.apache.catalina.core.Appli > >>>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:317) > >>>>> at org.springframework.security.w > >>>> eb.access.intercept.FilterSecurityInterceptor.invoke(FilterS > >>>> ecurityInterceptor.java:127) > >>>>> at org.springframework.security.w > >>>> eb.access.intercept.FilterSecurityInterceptor.doFilter(Filte > >>>> rSecurityInterceptor.java:91) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.access.ExceptionTranslationFilter.doFilter(ExceptionTrans > >>>> lationFilter.java:114) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.session.SessionManagementFilter.doFilter(SessionManagemen > >>>> tFilter.java:137) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.authentication.AnonymousAuthenticationFilter.doFilter(Ano > >>>> nymousAuthenticationFilter.java:111) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.servletapi.SecurityContextHolderAwareRequestFilter.doFilt > >>>> er(SecurityContextHolderAwareRequestFilter.java:170) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.savedrequest.RequestCacheAwareFilter.doFilter(RequestCach > >>>> eAwareFilter.java:63) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.authentication.www.BasicAuthenticationFilter.doFilterInte > >>>> rnal(BasicAuthenticationFilter.java:158) > >>>>> at org.springframework.web.filter > >>>> .OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.authentication.AbstractAuthenticationProcessingFilter.doF > >>>> ilter(AbstractAuthenticationProcessingFilter.java:200) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFi > >>>> lter.java:64) > >>>>> > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFi > >>>> lter.java:64) > >>>>> at org.springframework.web.filter > >>>> .OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.context.request.async.WebAsyncManagerIntegrationFilter.do > >>>> FilterInternal(WebAsyncManagerIntegrationFilter.java:56) > >>>>> at org.springframework.web.filter > >>>> .OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.context.SecurityContextPersistenceFilter.doFilter(Securit > >>>> yContextPersistenceFilter.java:105) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP > >>>> roxy.java:331) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) > >>>>> at org.springframework.security.w > >>>> eb.FilterChainProxy.doFilter(FilterChainProxy.java:177) > >>>>> at org.springframework.web.filter > >>>> .DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) > >>>>> at org.springframework.web.filter > >>>> .DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) > >>>>> at org.apache.catalina.core.Appli > >>>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > >>>>> at org.apache.catalina.core.Appli > >>>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) > >>>>> at com.thetransactioncompany.cors > >>>> .CORSFilter.doFilter(CORSFilter.java:209) > >>>>> at com.thetransactioncompany.cors > >>>> .CORSFilter.doFilter(CORSFilter.java:244) > >>>>> at org.apache.catalina.core.Appli > >>>> cationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > >>>>> at org.apache.catalina.core.Appli > >>>> cationFilterChain.doFilter(ApplicationFilterChain.java:208) > >>>>> at org.apache.catalina.core.Stand > >>>> ardWrapperValve.invoke(StandardWrapperValve.java:219) > >>>>> at org.apache.catalina.core.Stand > >>>> ardContextValve.invoke(StandardContextValve.java:110) > >>>>> at org.apache.catalina.core.Stand > >>>> ardHostValve.invoke(StandardHostValve.java:169) > >>>>> at org.apache.catalina.valves.Err > >>>> orReportValve.invoke(ErrorReportValve.java:103) > >>>>> at org.apache.catalina.valves.Acc > >>>> essLogValve.invoke(AccessLogValve.java:962) > >>>>> at org.apache.catalina.core.Stand > >>>> ardEngineValve.invoke(StandardEngineValve.java:116) > >>>>> at org.apache.catalina.connector. > >>>> CoyoteAdapter.service(CoyoteAdapter.java:445) > >>>>> at org.apache.coyote.http11.Abstr > >>>> actHttp11Processor.process(AbstractHttp11Processor.java:1115) > >>>>> at org.apache.coyote.AbstractProt > >>>> ocol$AbstractConnectionHandler.process(AbstractProtocol.java:637) > >>>>> at org.apache.tomcat.util.net.JIo > >>>> Endpoint$SocketProcessor.run(JIoEndpoint.java:316) > >>>>> at java.util.concurrent.ThreadPoo > >>>> lExecutor.runWorker(ThreadPoolExecutor.java:1149) > >>>>> at java.util.concurrent.ThreadPoo > >>>> lExecutor$Worker.run(ThreadPoolExecutor.java:624) > >>>>> at org.apache.tomcat.util.threads > >>>> .TaskThread$WrappingRunnable.run(TaskThread.java:61) > >>>>> at java.lang.Thread.run(Thread.java:748) > >>>> > >>> > >>> > >> > >
