BTW: KYLIN-2165 will handle this in a more elegant way, it will be released in next version;
2016-12-22 9:21 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>: > "java.io.FileNotFoundException: File does not exist: > /young/kylin_test/kylin_metadata_test/kylin-678266c0-ba0e- > 48b4-bdb5-6e578320375a/row_count/000000_0" > > Seems be the issue in https://issues.apache.org/jira/browse/KYLIN-2159 > It should be fixed in 1.6.0; could you please check whether there is any > file in the folder? > > hadoop fs -ls /young/kylin_test/kylin_metadata_test/kylin-678266c0- > ba0e-48b4-bdb5-6e578320375a/row_count/ > > 2016-12-20 15:23 GMT+08:00 Alberto Ramón <a.ramonporto...@gmail.com>: > >> other idea: >> Can be a problem with permissions?: the user that execute Kylin can't >> read data generated by YARN >> check if Kylin user can read your folder /young/kylin_test/ >> Which Hadoop user are executing Kylin? >> >> (no more ideas, Good Luck) >> >> 2016-12-20 7:51 GMT+01:00 雨日听风 <491245...@qq.com>: >> >>> Thank you! >>> We checked the yarn and hard disk. But not found any error. Hard disk >>> space and memory and so on is working well. >>> Last time its error code was "unknownhost clusterB",now in new server >>> env it cant find clusterB(hbase only). but cant find rowCount file. >>> === >>> the follow command runs ok: >>> hdfs dfs -mkdir /young/kylin_test/kylin_metadata_nokia/kylin >>> -678c15ba-5375-4f80-831e-1ae0af8ed576/row_count/tmp >>> And "ls" cant find file "000000_0" which it said "file does not exist". >>> >>> ------------------ 原始邮件 ------------------ >>> *发件人:* "Alberto Ramón";<a.ramonporto...@gmail.com>; >>> *发送时间:* 2016年12月19日(星期一) 晚上9:13 >>> *收件人:* "user"<user@kylin.apache.org>; >>> *主题:* Re: Error when #2 Step: Redistribute Flat Hive Table - File does >>> not exist >>> >>> i think i had this error last nigth :) >>> (go to yarn to find detailed error & find on internet) >>> in my case was free space less than 10% of hard disk. Check this please >>> >>> El 19/12/2016 11:35, "雨日听风" <491245...@qq.com> escribió: >>> >>>> When I build a cube in kylin1.6, I get error in step #2: Redistribute >>>> Flat Hive Table >>>> >>>> Please help! Thank you very much! >>>> >>>> env: kylin1.6 is in a independent server, and have 2 other server >>>> cluster: clusterA(hive only) and clusterB(hbase only). >>>> Error is: >>>> >>>> 2016-12-19 10:28:00,641 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Compute row count of flat hive >>>> table, cmd: >>>> 2016-12-19 10:28:00,642 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : hive -e "USE boco; >>>> SET dfs.replication=2; >>>> SET hive.exec.compress.output=true; >>>> SET hive.auto.convert.join.noconditionaltask=true; >>>> SET hive.auto.convert.join.noconditionaltask.size=100000000; >>>> SET mapreduce.output.fileoutputformat.compress.type=BLOCK; >>>> SET mapreduce.job.split.metainfo.maxsize=-1; >>>> SET mapreduce.job.queuename=young; >>>> SET tez.queue.name=young; >>>> >>>> set hive.exec.compress.output=false; >>>> >>>> set hive.exec.compress.output=false; >>>> INSERT OVERWRITE DIRECTORY '/young/kylin_test/kylin_metad >>>> ata_test/kylin-678266c0-ba0e-48b4-bdb5-6e578320375a/row_count' SELECT >>>> count(*) FROM kylin_intermediate_hbase_in_te >>>> stCluster_CUBE_f9468805_eabf_4b54_bf2b_182e4c86214a; >>>> >>>> " >>>> 2016-12-19 10:28:03,277 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : WARNING: Use "yarn jar" to launch >>>> YARN applications. >>>> 2016-12-19 10:28:04,444 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : >>>> 2016-12-19 10:28:04,445 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Logging initialized using >>>> configuration in file:/etc/hive/conf/hive-log4j.properties >>>> 2016-12-19 10:28:14,700 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : OK >>>> 2016-12-19 10:28:14,703 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Time taken: 0.935 seconds >>>> 2016-12-19 10:28:15,559 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Query ID = >>>> young_20161219102814_a7104fd4-ba83-47fc-ac0b-0c9bef4e1969 >>>> 2016-12-19 10:28:15,560 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Total jobs = 1 >>>> 2016-12-19 10:28:15,575 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Launching Job 1 out of 1 >>>> 2016-12-19 10:28:22,842 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : >>>> 2016-12-19 10:28:22,842 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : >>>> 2016-12-19 10:28:23,104 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Status: Running (Executing on YARN >>>> cluster with App id application_1473415773736_1063281) >>>> 2016-12-19 10:28:23,104 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : >>>> 2016-12-19 10:28:23,104 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Map 1: -/- Reducer 2: 0/1 >>>> 2016-12-19 10:28:23,307 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Map 1: 0/2 Reducer 2: 0/1 >>>> 2016-12-19 10:28:26,363 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Map 1: 0/2 Reducer 2: 0/1 >>>> 2016-12-19 10:28:26,567 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Map 1: 0(+1)/2 Reducer 2: 0/1 >>>> 2016-12-19 10:28:26,596 INFO [pool-7-thread-1] >>>> threadpool.DefaultScheduler:118 : Job Fetcher: 1 should running, 1 >>>> actual running, 0 ready, 0 already succeed, 3 error, 1 discarded, 0 others >>>> 2016-12-19 10:28:26,769 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Map 1: 0(+2)/2 Reducer 2: 0/1 >>>> 2016-12-19 10:28:29,810 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Map 1: 0(+2)/2 Reducer 2: 0/1 >>>> 2016-12-19 10:28:30,217 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Map 1: 1(+1)/2 Reducer 2: 0(+1)/1 >>>> 2016-12-19 10:28:30,826 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Map 1: 2/2 Reducer 2: 0(+1)/1 >>>> 2016-12-19 10:28:31,232 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Map 1: 2/2 Reducer 2: 1/1 >>>> 2016-12-19 10:28:31,319 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Moving data to: >>>> /young/kylin_test/kylin_metadata_test/kylin-678266c0-ba0e-48 >>>> b4-bdb5-6e578320375a/row_count >>>> 2016-12-19 10:28:31,406 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : OK >>>> 2016-12-19 10:28:31,454 INFO [pool-8-thread-7] >>>> execution.AbstractExecutable:36 : Time taken: 16.701 seconds >>>> 2016-12-19 10:28:35,074 ERROR [pool-8-thread-7] >>>> execution.AbstractExecutable:357 : >>>> job:678266c0-ba0e-48b4-bdb5-6e578320375a-01 >>>> execute finished with exception >>>> java.io.FileNotFoundException: File does not exist: >>>> /young/kylin_test/kylin_metadata_test/kylin-678266c0-ba0e-48 >>>> b4-bdb5-6e578320375a/row_count/000000_0 >>>> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INo >>>> deFile.java:71) >>>> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INo >>>> deFile.java:61) >>>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlock >>>> LocationsInt(FSNamesystem.java:1820) >>>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlock >>>> Locations(FSNamesystem.java:1791) >>>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlock >>>> Locations(FSNamesystem.java:1704) >>>> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.get >>>> BlockLocations(NameNodeRpcServer.java:587) >>>> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServ >>>> erSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolS >>>> erverSideTranslatorPB.java:365) >>>> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocol >>>> Protos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNam >>>> enodeProtocolProtos.java) >>>> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcIn >>>> voker.call(ProtobufRpcEngine.java:616) >>>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969) >>>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2081) >>>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2077) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at javax.security.auth.Subject.doAs(Subject.java:415) >>>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGro >>>> upInformation.java:1657) >>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2075) >>>> >>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>>> Method) >>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Native >>>> ConstructorAccessorImpl.java:57) >>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(De >>>> legatingConstructorAccessorImpl.java:45) >>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:526) >>>> at org.apache.hadoop.ipc.RemoteException.instantiateException(R >>>> emoteException.java:106) >>>> at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException( >>>> RemoteException.java:73) >>>> at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSCl >>>> ient.java:1242) >>>> at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient. >>>> java:1227) >>>> at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient. >>>> java:1215) >>>> at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndG >>>> etLastBlockLength(DFSInputStream.java:303) >>>> at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStrea >>>> m.java:269) >>>> at org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream. >>>> java:261) >>>> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1540) >>>> at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(Distri >>>> butedFileSystem.java:303) >>>> at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(Distri >>>> butedFileSystem.java:299) >>>> at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSyst >>>> emLinkResolver.java:81) >>>> at org.apache.hadoop.hdfs.DistributedFileSystem.open(Distribute >>>> dFileSystem.java:299) >>>> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767) >>>> at org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiv >>>> eTableStep.readRowCountFromFile(HiveMRInput.java:294) >>>> at org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiv >>>> eTableStep.doWork(HiveMRInput.java:337) >>>> at org.apache.kylin.job.execution.AbstractExecutable.execute(Ab >>>> stractExecutable.java:113) >>>> at org.apache.kylin.job.execution.DefaultChainedExecutable.doWo >>>> rk(DefaultChainedExecutable.java:57) >>>> at org.apache.kylin.job.execution.AbstractExecutable.execute(Ab >>>> stractExecutable.java:113) >>>> at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRun >>>> ner.run(DefaultScheduler.java:136) >>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >>>> Executor.java:1145) >>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >>>> lExecutor.java:615) >>>> at java.lang.Thread.run(Thread.java:745) >>>> Caused by: >>>> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): >>>> File does not exist: /young/kylin_test/kylin_metada >>>> ta_test/kylin-678266c0-ba0e-48b4-bdb5-6e578320375a/row_count/000000_0 >>>> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INo >>>> deFile.java:71) >>>> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INo >>>> deFile.java:61) >>>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlock >>>> LocationsInt(FSNamesystem.java:1820) >>>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlock >>>> Locations(FSNamesystem.java:1791) >>>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlock >>>> Locations(FSNamesystem.java:1704) >>>> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.get >>>> BlockLocations(NameNodeRpcServer.java:587) >>>> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServ >>>> erSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolS >>>> erverSideTranslatorPB.java:365) >>>> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocol >>>> Protos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNam >>>> enodeProtocolProtos.java) >>>> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcIn >>>> voker.call(ProtobufRpcEngine.java:616) >>>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969) >>>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2081) >>>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2077) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at javax.security.auth.Subject.doAs(Subject.java:415) >>>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGro >>>> upInformation.java:1657) >>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2075) >>>> >>>> at org.apache.hadoop.ipc.Client.call(Client.java:1427) >>>> at org.apache.hadoop.ipc.Client.call(Client.java:1358) >>>> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(Proto >>>> bufRpcEngine.java:229) >>>> at com.sun.proxy.$Proxy25.getBlockLocations(Unknown Source) >>>> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTran >>>> slatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:255) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >>>> ssorImpl.java:57) >>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >>>> thodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMeth >>>> od(RetryInvocationHandler.java:252) >>>> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(Ret >>>> ryInvocationHandler.java:104) >>>> at com.sun.proxy.$Proxy26.getBlockLocations(Unknown Source) >>>> at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSCl >>>> ient.java:1240) >>>> ... 20 more >>>> 2016-12-19 10:28:35,078 DEBUG [pool-8-thread-7] dao.ExecutableDao:210 : >>>> updating job output, id: 678266c0-ba0e-48b4-bdb5-6e578320375a-01 >>>> 2016-12-19 10:28:35,082 DEBUG [pool-8-thread-7] >>>> hbase.HBaseResourceStore:262 : Update row >>>> /execute_output/678266c0-ba0e-48b4-bdb5-6e578320375a-01 >>>> from oldTs: 1482114480638, to newTs: 1482114515078, operation result: true >>>> 2016-12-19 10:28:35,085 DEBUG [pool-8-thread-7] dao.ExecutableDao:210 : >>>> updating job output, id: 678266c0-ba0e-48b4-bdb5-6e578320375a-01 >>>> >>> >> > > > -- > Best regards, > > Shaofeng Shi 史少锋 > > -- Best regards, Shaofeng Shi 史少锋
