Re: ERROR WHEN Saving Cuboid Statistics
The missing > hdfs:/kylin/kylin_metadata/kylin-6faa47ea-3a5b-4020-976f-c9fcf9d93bd2/kylin_sales_cube/fact_distinct_columns/statistics should be the output of the "Fact distinct values" step. I would check 1) if the file is really there; 2) in kylin.log if anything suspicious around the step of "Fact distinct values". If find nothing, then discard and re-submit the job is a temporary workaround. On Fri, Jun 8, 2018 at 11:08 AM, op <520075...@qq.com> wrote: > > Hi,everyone,I‘am running example kylin_sales_model. > > At saving cuboid statistics step > >log says > >details in kylin.log are below: > > there isn't any other error information ,and i can't find out what's > wrong,can you help me please? > my version : > kylin 2.0.0 > hbase 1.2.6 > hdaoop 2.9.0 > hive 2.3.2 beeline > > 2018-06-07 16:08:18,252 WARN [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] > conf.HiveConf:4116 : HiveConf of name hive.mapred.supports.subdirectories > does not exist > 2018-06-07 16:08:18,253 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] > hive.metastore:407 : Trying to connect to metastore with URI > thrift://hadoop1001:9083 > 2018-06-07 16:08:18,254 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] > hive.metastore:481 : Opened a connection to metastore, current connections: > 51 > 2018-06-07 16:08:18,254 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] > hive.metastore:534 : Connected to metastore. > 2018-06-07 16:08:18,341 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] > lookup.SnapshotManager:214 : Loading snapshotTable from > /table_snapshot/KYLIN.KYLIN_CAL_DT/c611bd70-b196-45ef-9edc-18d6a1126277.snapshot, > with loadData: false > 2018-06-07 16:08:18,384 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] > lookup.SnapshotManager:130 : Identical input FileSignature > [path=hdfs://beh/user/hive/warehouse/kylin.db/kylin_cal_dt, size=512997, > lastModifiedTime=1528356298671], reuse existing snapshot at > /table_snapshot/KYLIN.KYLIN_CAL_DT/c611bd70-b196-45ef-9edc-18d6a1126277 > .snapshot > 2018-06-07 16:08:18,384 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] > cube.CubeManager:366 : Updating cube instance 'kylin_sales_cube' > 2018-06-07 16:08:18,384 WARN [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] > cube.CubeValidator:102 : NEW segment start does not fit/connect with other > segments: kylin_sales_cube[2012010100_2012123100] > 2018-06-07 16:08:18,384 WARN [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] > cube.CubeValidator:104 : NEW segment end does not fit/connect with other > segments: kylin_sales_cube[2012010100_2012123100] > 2018-06-07 16:08:18,387 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] > cli.DictionaryGeneratorCLI:91 : Building snapshot of KYLIN.KYLIN_COUNTRY > 2018-06-07 16:08:18,387 INFO [pool-11-thread-1] > cachesync.Broadcaster:123 : Announcing new broadcast event: > BroadcastEvent{entity=cube, event=update, cacheKey=kylin_sales_cube} > 2018-06-07 16:08:18,391 DEBUG [http-bio-7070-exec-3] > cachesync.Broadcaster:189 : Broadcasting metadata change: entity=cube, > event=UPDATE, cacheKey=kylin_sales_cube, listeners=[org.apache.kylin. > cube.CubeManager$CubeSyncListener@11f9b854, org.apache.kylin.storage. > hybrid.HybridManager$HybridSyncListener@581a3b84, > org.apache.kylin.rest.service.CacheService$1@6c4e4197] > 2018-06-07 16:08:18,393 INFO [http-bio-7070-exec-3] cube.CubeManager:785 > : Reloaded cube kylin_sales_cube being CUBE[name=kylin_sales_cube] having 1 > segments > 2018-06-07 16:08:18,393 DEBUG [http-bio-7070-exec-3] > cachesync.Broadcaster:189 : Broadcasting metadata change: > entity=project_data, event=UPDATE, cacheKey=learn_kylin, > listeners=[org.apache.kylin.metadata.project.ProjectManager$ > ProjectSyncListener@51853c91, org.apache.kylin.metadata.MetadataManager$ > SrcTableSyncListener@67cc135, org.apache.kylin.metadata.MetadataManager$ > SrcTableExtSyncListener@33eb1107, org.apache.kylin.metadata. > MetadataManager$DataModelSyncListener@6d749bd4, org.apache.kylin.metadata. > MetadataManager$ExtFilterSyncListener@9f57e2e, org.apache.kylin.cube. > CubeDescManager$CubeDescSyncListener@71ecba35, org.apache.kylin.cube. > CubeManager$CubeSyncListener@11f9b854, org.apache.kylin.storage. > hybrid.HybridManager$HybridSyncListener@581a3b84, > org.apache.kylin.metadata.realization.RealizationRegistry$1@7a3c2aa7, > org.apache.kylin.rest.service.CacheService$1@6c4e4197] > 2018-06-07 16:08:18,394 INFO [http-bio-7070-exec-3] > service.CacheService:135 : removeOLAPDataSource is called for project > learn_kylin > 2018-06-07 16:08:18,394 INFO [http-bio-7070-exec-3] > service.CacheService:117 : cleaning cache for project learn_kylin > (currently remove all entries) > 2018-06-07 16:08:18,394 DEBUG [http-bio-7070-exec-3] > cachesync.Broadcaster:219 : Done broadcasting metadata change: > entity=project_data, event=UPDATE, cacheKey=learn_kylin > 2018-06-07 16:08:18,395 DEBUG [http-bio-7070-exec-3] > cachesync.Broadcaster:219 : Done broadcasting metada
ERROR WHEN Saving Cuboid Statistics
Hi??everyone,I??am running example kylin_sales_model. At saving cuboid statistics step log says details in kylin.log are below: there isn't any other error information ,and i can't find out what's wrong,can you help me please? my version : kylin 2.0.0 hbase 1.2.6 hdaoop 2.9.0 hive 2.3.2 beeline 2018-06-07 16:08:18,252 WARN [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] conf.HiveConf:4116 : HiveConf of name hive.mapred.supports.subdirectories does not exist 2018-06-07 16:08:18,253 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] hive.metastore:407 : Trying to connect to metastore with URI thrift://hadoop1001:9083 2018-06-07 16:08:18,254 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] hive.metastore:481 : Opened a connection to metastore, current connections: 51 2018-06-07 16:08:18,254 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] hive.metastore:534 : Connected to metastore. 2018-06-07 16:08:18,341 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] lookup.SnapshotManager:214 : Loading snapshotTable from /table_snapshot/KYLIN.KYLIN_CAL_DT/c611bd70-b196-45ef-9edc-18d6a1126277.snapshot, with loadData: false 2018-06-07 16:08:18,384 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] lookup.SnapshotManager:130 : Identical input FileSignature [path=hdfs://beh/user/hive/warehouse/kylin.db/kylin_cal_dt, size=512997, lastModifiedTime=1528356298671], reuse existing snapshot at /table_snapshot/KYLIN.KYLIN_CAL_DT/c611bd70-b196-45ef-9edc-18d6a1126277.snapshot 2018-06-07 16:08:18,384 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] cube.CubeManager:366 : Updating cube instance 'kylin_sales_cube' 2018-06-07 16:08:18,384 WARN [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] cube.CubeValidator:102 : NEW segment start does not fit/connect with other segments: kylin_sales_cube[2012010100_2012123100] 2018-06-07 16:08:18,384 WARN [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] cube.CubeValidator:104 : NEW segment end does not fit/connect with other segments: kylin_sales_cube[2012010100_2012123100] 2018-06-07 16:08:18,387 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] cli.DictionaryGeneratorCLI:91 : Building snapshot of KYLIN.KYLIN_COUNTRY 2018-06-07 16:08:18,387 INFO [pool-11-thread-1] cachesync.Broadcaster:123 : Announcing new broadcast event: BroadcastEvent{entity=cube, event=update, cacheKey=kylin_sales_cube} 2018-06-07 16:08:18,391 DEBUG [http-bio-7070-exec-3] cachesync.Broadcaster:189 : Broadcasting metadata change: entity=cube, event=UPDATE, cacheKey=kylin_sales_cube, listeners=[org.apache.kylin.cube.CubeManager$CubeSyncListener@11f9b854, org.apache.kylin.storage.hybrid.HybridManager$HybridSyncListener@581a3b84, org.apache.kylin.rest.service.CacheService$1@6c4e4197] 2018-06-07 16:08:18,393 INFO [http-bio-7070-exec-3] cube.CubeManager:785 : Reloaded cube kylin_sales_cube being CUBE[name=kylin_sales_cube] having 1 segments 2018-06-07 16:08:18,393 DEBUG [http-bio-7070-exec-3] cachesync.Broadcaster:189 : Broadcasting metadata change: entity=project_data, event=UPDATE, cacheKey=learn_kylin, listeners=[org.apache.kylin.metadata.project.ProjectManager$ProjectSyncListener@51853c91, org.apache.kylin.metadata.MetadataManager$SrcTableSyncListener@67cc135, org.apache.kylin.metadata.MetadataManager$SrcTableExtSyncListener@33eb1107, org.apache.kylin.metadata.MetadataManager$DataModelSyncListener@6d749bd4, org.apache.kylin.metadata.MetadataManager$ExtFilterSyncListener@9f57e2e, org.apache.kylin.cube.CubeDescManager$CubeDescSyncListener@71ecba35, org.apache.kylin.cube.CubeManager$CubeSyncListener@11f9b854, org.apache.kylin.storage.hybrid.HybridManager$HybridSyncListener@581a3b84, org.apache.kylin.metadata.realization.RealizationRegistry$1@7a3c2aa7, org.apache.kylin.rest.service.CacheService$1@6c4e4197] 2018-06-07 16:08:18,394 INFO [http-bio-7070-exec-3] service.CacheService:135 : removeOLAPDataSource is called for project learn_kylin 2018-06-07 16:08:18,394 INFO [http-bio-7070-exec-3] service.CacheService:117 : cleaning cache for project learn_kylin (currently remove all entries) 2018-06-07 16:08:18,394 DEBUG [http-bio-7070-exec-3] cachesync.Broadcaster:219 : Done broadcasting metadata change: entity=project_data, event=UPDATE, cacheKey=learn_kylin 2018-06-07 16:08:18,395 DEBUG [http-bio-7070-exec-3] cachesync.Broadcaster:219 : Done broadcasting metadata change: entity=cube, event=UPDATE, cacheKey=kylin_sales_cube 2018-06-07 16:08:18,511 WARN [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] conf.HiveConf:4116 : HiveConf of name hive.server2.custom.authentication.file does not exist 2018-06-07 16:08:18,512 WARN [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] conf.HiveConf:4116 : HiveConf of name hive.mapred.supports.subdirectories does not exist 2018-06-07 16:08:18,513 INFO [Job 6faa47ea-3a5b-4020-976f-c9fcf9d93bd2-424] hive.metastore:407 : Trying to connect to metastore with URI thrift://hadoop1001:9083 2018-06-07 16:0