it's case when kylin read orc format table,if the table is textfile format,is can work.
| | 王廉鑫 | | [email protected] | 签名由网易邮箱大师定制 On 9/10/2019 17:25，Yaqian Zhang<[email protected]> wrote： Hi 廉鑫： This artical maybe could resolve your problem. https://stackoverflow.com/questions/35075276/apache-kylin-intermediate-table-not-found On Sep 9, 2019, at 5:55 PM, 王廉鑫 <[email protected]> wrote: Hi, kylin2.6.2 on FusionInsight C.6.5.1 when Cbur run 4 Step. it get eror!!! . 2019-09-09 17:51:31,538 INFO [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511] common.HCatUtil:638 : Configuration differences={hive.metastore.sasl.enabled=true, hive.metastore.try.direct.sql.ddl=false, hive.metastore.kerberos.principal=hive/[email protected], javax.jdo.option.ConnectionURL=jdbc:postgresql://10.10.20.96:20051/hive${SERVICE_INDEX}meta?socketTimeout=60, hive.server2.thrift.sasl.qop=auth-conf, hive.mapred.reduce.tasks.speculative.execution=false, javax.jdo.option.ConnectionUserName=hive, hive.server2.authentication.kerberos.principal=hive/[email protected], javax.jdo.option.ConnectionPassword=d2NjX2NyeXB0ATQxNDU1MzVGNDM0MjQzOzQxNDEzODQxNDEzNDQzMzkzNDM2Mzc0MTMyMzYzNTMzMzI0MjM4NDY0NDQyNDEzNDQzMzAzNjM4NDMzOTMxNDM7OzMyMzUzMDMwOzRFMjhDRkQyNTlDOTMxRTA5MjA2QUU2MzY2QzRCQzZBO0JFQTBDMzgyRDU1RkJGMzM7NjQzNjMxMzMzNDM3NjY2NTJEMzkzNTM1NjQyRDM0MzQzODM5MkQzOTYyMzczMzJENjU2NjM4MzU2MzMwMzgzMTM3MzY2NjYzOw, hive.exec.scratchdir=/tmp/hive-scratch, javax.jdo.dbuser.decode.enable=true, hive.keychain.dir=/user/hive/keychain, hive.metastore.metrics.enabled=true, hive.metastore.uris=thrift://10.10.20.94:21088,thrift://10.10.20.91:21088, hive.server2.authentication=KERBEROS, javax.jdo.option.ConnectionDriverName=org.postgresql.Driver, hive.server2.authentication.kerberos.keytab=/opt/huawei/Bigdata/FusionInsight_HD_6.5.1/install/FusionInsight-Hive-3.1.0/hive-3.1.0/bin/hive.keytab} 2019-09-09 17:51:31,540 INFO [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511] metastore.HiveMetaStoreClient:535 : Trying to connect to metastore with URI thrift://RDC-BD11:21088 2019-09-09 17:51:31,541 INFO [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511] metastore.HiveMetaStoreClient:608 : HMSC::open(): Could not find delegation token. Creating KERBEROS-based thrift connection. 2019-09-09 17:51:31,547 INFO [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511] metastore.HiveMetaStoreClient:635 : Opened a connection to metastore, current connections: 1 2019-09-09 17:51:31,548 INFO [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511] metastore.HiveMetaStoreClient:740 : Connected to metastore. 2019-09-09 17:51:31,548 INFO [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511] metastore.RetryingMetaStoreClient:98 : RetryingMetaStoreClient proxy=class org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient [email protected] (auth:KERBEROS) retries=1 delay=1 lifetime=0 2019-09-09 17:51:31,565 WARN [Scheduler 1464751025 Job d780b707-aebf-802b-bf84-dc96805d8a10-367] model.Segments:444 : NEW segment start does not fit/connect with other segments: GL1221160200CAMT[20190901000000_20190902000000] 2019-09-09 17:51:31,566 ERROR [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511] common.MapReduceExecutable:138 : error execute MapReduceExecutable{id=23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-00, name=null, state=RUNNING} java.lang.RuntimeException: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT table not found) at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:83) at org.apache.kylin.source.hive.cardinality.HiveColumnCardinalityJob.run(HiveColumnCardinalityJob.java:93) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:131) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT table not found) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51) at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:80) ... 9 more Caused by: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result.read(ThriftHiveMetastore.java) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_req(ThriftHiveMetastore.java:2113) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_req(ThriftHiveMetastore.java:2100) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1753) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1745) at sun.reflect.GeneratedMethodAccessor158.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:209) at com.sun.proxy.$Proxy77.getTable(Unknown Source) at org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:191) at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:105) at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:88) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95) ... 11 more 2019-09-09 17:51:31,566 WARN [Scheduler 1464751025 Job d780b707-aebf-802b-bf84-dc96805d8a10-367] model.Segments:446 : NEW segment end does not fit/connect with other segments: GL1221160200CAMT[20190901000000_20190902000000] 2019-09-09 17:51:31,583 ERROR [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511] execution.AbstractExecutable:169 : error running Executable: DefaultChainedExecutable{id=23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b, name=Hive Column Cardinality calculation for table 'METRIC_V2_AIA.GL1221160200CAMT', state=RUNNING} 2019-09-09 17:51:31,600 ERROR [pool-12-thread-10] threadpool.DefaultScheduler:116 : ExecuteException job:23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.exception.ExecuteException: java.lang.RuntimeException: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT table not found) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:182) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.kylin.job.exception.ExecuteException: java.lang.RuntimeException: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT table not found) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:182) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) ... 4 more Caused by: java.lang.RuntimeException: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT table not found) at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:83) at org.apache.kylin.source.hive.cardinality.HiveColumnCardinalityJob.run(HiveColumnCardinalityJob.java:93) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:131) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) ... 6 more Caused by: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT table not found) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51) at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:80) ... 9 more Caused by: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result.read(ThriftHiveMetastore.java) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_req(ThriftHiveMetastore.java:2113) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_req(ThriftHiveMetastore.java:2100) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1753) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1745) at sun.reflect.GeneratedMethodAccessor158.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:209) at com.sun.proxy.$Proxy77.getTable(Unknown Source) at org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:191) at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:105) at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:88) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95) ... 11 more 2019-09-09 17:51:31,652 WARN [Scheduler 1464751025 Job d780b707-aebf-802b-bf84-dc96805d8a10-367] model.Segments:444 : NEW segment start does not fit/connect with other segments: GL1221160200CAMT[20190901000000_20190902000000] | [email protected] | 签名由网易邮箱大师定制
