This appears to be a Hive issue (something probably called FS.close() too early?). Redirecting to the Hive user lists as they can help better with this.
On Tue, Jul 10, 2012 at 9:59 PM, 안의건 <ahneui...@gmail.com> wrote: > Hello. I have a problem with the filesystem closing. > > The filesystem was closed when the hive query is running. > It is 'select' query and the data size is about 1TB. > I'm using hadoop-0.20.2 and hive-0.7.1. > > The error log is telling that tmp file is not deleted, or the tmp path > exception is occurred. > > Is there any hadoop configuration I'm missing? > > Thank you > > [stderr logs] > org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: > Filesystem closed > at > org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:454) > at > org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:636) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:557) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) > at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > at org.apache.hadoop.mapred.Child.main(Child.java:170) > Caused by: java.io.IOException: Filesystem closed > at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:226) > at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:617) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:453) > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:648) > at org.apache.hadoop.fs.FileSystem.deleteOnExit(FileSystem.java:615) > at > org.apache.hadoop.hive.shims.Hadoop20Shims.fileSystemDeleteOnExit(Hadoop20Shims.java:68) > at > org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:451) > ... 12 more -- Harsh J
