[jira] Commented: (HIVE-1326) RowContainer uses hard-coded '/tmp/' path for temporary files
[ https://issues.apache.org/jira/browse/HIVE-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861188#action_12861188 ] Ning Zhang commented on HIVE-1326: -- I guess this is due to https://issues.apache.org/jira/browse/HADOOP-2735 which set java.io.tmpdir to be dependent on mapred.tmp.dir. In that case the change of using createTempFile looks fine with me. Also can you remove the parentFile.delete() in your patch? If we set java.io.tmpdir to work/tmp then we don't have file name conflicts, But if the user choose to use shared /tmp then it may remove files created by other tasks. Yongqiang, any further comments? > RowContainer uses hard-coded '/tmp/' path for temporary files > - > > Key: HIVE-1326 > URL: https://issues.apache.org/jira/browse/HIVE-1326 > Project: Hadoop Hive > Issue Type: Bug > Environment: Hadoop 0.19.2 with Hive trunk. We're using FreeBSD 7.0, > but that doesn't seem relevant. >Reporter: Michael Klatt > Attachments: rowcontainer.patch > > > In our production hadoop environment, the "/tmp/" is actually pretty small, > and we encountered a problem when a query used the RowContainer class and > filled up the /tmp/ partition. I tracked down the cause to the RowContainer > class putting temporary files in the '/tmp/' path instead of using the > configured Hadoop temporary path. I've attached a patch to fix this. > Here's the traceback: > 2010-04-25 12:05:05,120 INFO > org.apache.hadoop.hive.ql.exec.persistence.RowContainer: RowContainer created > temp file /tmp/hive-rowcontainer-1244151903/RowContainer7816.tmp > 2010-04-25 12:05:06,326 INFO ExecReducer: ExecReducer: processing 1000 > rows: used memory = 385520312 > 2010-04-25 12:05:08,513 INFO ExecReducer: ExecReducer: processing 1100 > rows: used memory = 341780472 > 2010-04-25 12:05:10,697 INFO ExecReducer: ExecReducer: processing 1200 > rows: used memory = 301446768 > 2010-04-25 12:05:12,837 INFO ExecReducer: ExecReducer: processing 1300 > rows: used memory = 399208768 > 2010-04-25 12:05:15,085 INFO ExecReducer: ExecReducer: processing 1400 > rows: used memory = 364507216 > 2010-04-25 12:05:17,260 INFO ExecReducer: ExecReducer: processing 1500 > rows: used memory = 332907280 > 2010-04-25 12:05:19,580 INFO ExecReducer: ExecReducer: processing 1600 > rows: used memory = 298774096 > 2010-04-25 12:05:21,629 INFO ExecReducer: ExecReducer: processing 1700 > rows: used memory = 396505408 > 2010-04-25 12:05:23,830 INFO ExecReducer: ExecReducer: processing 1800 > rows: used memory = 362477288 > 2010-04-25 12:05:25,914 INFO ExecReducer: ExecReducer: processing 1900 > rows: used memory = 327229744 > 2010-04-25 12:05:27,978 INFO ExecReducer: ExecReducer: processing 2000 > rows: used memory = 296051904 > 2010-04-25 12:05:28,155 FATAL ExecReducer: org.apache.hadoop.fs.FSError: > java.io.IOException: No space left on device > at > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:199) > at > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) > at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109) > at > org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49) > at java.io.DataOutputStream.write(DataOutputStream.java:90) > at > org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.writeChunk(ChecksumFileSystem.java:346) > at > org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:150) > at > org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:132) > at > org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:121) > at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:112) > at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86) > at > org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49) > at java.io.DataOutputStream.write(DataOutputStream.java:90) > at > org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1013) > at > org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977) > at > org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat$1.write(HiveSequenceFileOutputFormat.java:70) > at > org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:343) > at > org.apache.hadoop.hive.ql.exec.persistence.RowContainer.add(RowContainer.java:163) > at > org.apache.hadoop.hive.ql.exec.JoinOperator.processOp(JoinOperator.java:118) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:456) > at > org.apache.hadoop.hive.ql.exec.ExecReducer.r
[jira] Updated: (HIVE-1325) dynamic partition insert should throw an exception if the number of target table columns + dynamic partition columns does not equal to the number of select columns
[ https://issues.apache.org/jira/browse/HIVE-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1325: - Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed Committed. Thanks Ning > dynamic partition insert should throw an exception if the number of target > table columns + dynamic partition columns does not equal to the number of > select columns > --- > > Key: HIVE-1325 > URL: https://issues.apache.org/jira/browse/HIVE-1325 > Project: Hadoop Hive > Issue Type: Bug >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1325.2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1321) bugs with temp directories, trailing blank fields in HBase bulk load
[ https://issues.apache.org/jira/browse/HIVE-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1321: - Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed Committed. Thanks John > bugs with temp directories, trailing blank fields in HBase bulk load > > > Key: HIVE-1321 > URL: https://issues.apache.org/jira/browse/HIVE-1321 > Project: Hadoop Hive > Issue Type: Bug > Components: HBase Handler >Affects Versions: 0.6.0 >Reporter: John Sichi >Assignee: John Sichi > Fix For: 0.6.0 > > Attachments: HIVE-1321.1.patch > > > HIVE-1295 had two bugs discovered during testing with production data: > (1) extra directories may be present in the output directory depending on how > the cluster is configured; we need to walk down these to find the column > family directory > (2) if a record ends with fields which are blank strings, the text format > omits the corresponding Control-A delimiters, so we need to fill in blanks > for these fields (instead of throwing ArrayIndexOutOfBoundsException) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1325) dynamic partition insert should throw an exception if the number of target table columns + dynamic partition columns does not equal to the number of select columns
[ https://issues.apache.org/jira/browse/HIVE-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861127#action_12861127 ] Namit Jain commented on HIVE-1325: -- +1 looks good - will commit if the tests pass > dynamic partition insert should throw an exception if the number of target > table columns + dynamic partition columns does not equal to the number of > select columns > --- > > Key: HIVE-1325 > URL: https://issues.apache.org/jira/browse/HIVE-1325 > Project: Hadoop Hive > Issue Type: Bug >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1325.2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1325) dynamic partition insert should throw an exception if the number of target table columns + dynamic partition columns does not equal to the number of select columns
[ https://issues.apache.org/jira/browse/HIVE-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861125#action_12861125 ] Namit Jain commented on HIVE-1325: -- I will take a look > dynamic partition insert should throw an exception if the number of target > table columns + dynamic partition columns does not equal to the number of > select columns > --- > > Key: HIVE-1325 > URL: https://issues.apache.org/jira/browse/HIVE-1325 > Project: Hadoop Hive > Issue Type: Bug >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1325.2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1326) RowContainer uses hard-coded '/tmp/' path for temporary files
[ https://issues.apache.org/jira/browse/HIVE-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861080#action_12861080 ] Michael Klatt commented on HIVE-1326: - In my brief experiments, I found that File.createTempFile() used the work/tmp path for the individual task (in Hadoop). I suspect that the java.io.tmpdir is being set by the TaskTracker. This has the advantage of letting Hadoop do the temporary file cleanup when the task cleanup happens. I think it would be great if this was the default behavior, but admins could override that with a hive specific parameter. I don't have any attachment to the actual implementation. I can spend more time to produce a different patch if needed, but I suspected someone would want to implement it differently. > RowContainer uses hard-coded '/tmp/' path for temporary files > - > > Key: HIVE-1326 > URL: https://issues.apache.org/jira/browse/HIVE-1326 > Project: Hadoop Hive > Issue Type: Bug > Environment: Hadoop 0.19.2 with Hive trunk. We're using FreeBSD 7.0, > but that doesn't seem relevant. >Reporter: Michael Klatt > Attachments: rowcontainer.patch > > > In our production hadoop environment, the "/tmp/" is actually pretty small, > and we encountered a problem when a query used the RowContainer class and > filled up the /tmp/ partition. I tracked down the cause to the RowContainer > class putting temporary files in the '/tmp/' path instead of using the > configured Hadoop temporary path. I've attached a patch to fix this. > Here's the traceback: > 2010-04-25 12:05:05,120 INFO > org.apache.hadoop.hive.ql.exec.persistence.RowContainer: RowContainer created > temp file /tmp/hive-rowcontainer-1244151903/RowContainer7816.tmp > 2010-04-25 12:05:06,326 INFO ExecReducer: ExecReducer: processing 1000 > rows: used memory = 385520312 > 2010-04-25 12:05:08,513 INFO ExecReducer: ExecReducer: processing 1100 > rows: used memory = 341780472 > 2010-04-25 12:05:10,697 INFO ExecReducer: ExecReducer: processing 1200 > rows: used memory = 301446768 > 2010-04-25 12:05:12,837 INFO ExecReducer: ExecReducer: processing 1300 > rows: used memory = 399208768 > 2010-04-25 12:05:15,085 INFO ExecReducer: ExecReducer: processing 1400 > rows: used memory = 364507216 > 2010-04-25 12:05:17,260 INFO ExecReducer: ExecReducer: processing 1500 > rows: used memory = 332907280 > 2010-04-25 12:05:19,580 INFO ExecReducer: ExecReducer: processing 1600 > rows: used memory = 298774096 > 2010-04-25 12:05:21,629 INFO ExecReducer: ExecReducer: processing 1700 > rows: used memory = 396505408 > 2010-04-25 12:05:23,830 INFO ExecReducer: ExecReducer: processing 1800 > rows: used memory = 362477288 > 2010-04-25 12:05:25,914 INFO ExecReducer: ExecReducer: processing 1900 > rows: used memory = 327229744 > 2010-04-25 12:05:27,978 INFO ExecReducer: ExecReducer: processing 2000 > rows: used memory = 296051904 > 2010-04-25 12:05:28,155 FATAL ExecReducer: org.apache.hadoop.fs.FSError: > java.io.IOException: No space left on device > at > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:199) > at > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) > at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109) > at > org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49) > at java.io.DataOutputStream.write(DataOutputStream.java:90) > at > org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.writeChunk(ChecksumFileSystem.java:346) > at > org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:150) > at > org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:132) > at > org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:121) > at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:112) > at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86) > at > org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49) > at java.io.DataOutputStream.write(DataOutputStream.java:90) > at > org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1013) > at > org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977) > at > org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat$1.write(HiveSequenceFileOutputFormat.java:70) > at > org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:343) > at > org.apache.hadoop.hive.ql.exec.persistence.RowContainer.add(RowContainer.java:163) > at > org.apache.hadoop.hive.ql.exec.JoinOperator.processOp(JoinOperator
[jira] Commented: (HIVE-1326) RowContainer uses hard-coded '/tmp/' path for temporary files
[ https://issues.apache.org/jira/browse/HIVE-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861054#action_12861054 ] Ning Zhang commented on HIVE-1326: -- The general idea of why we put temp files and directories under /tmp is two fold: 1) it allows easy clean up in case of any unexpected interruptions to the JVM. In the regular exits, these temp files/directories should be removed by JVM by deleteOnExit(), but if the JVM was killed/interrupted unexpected, these temp files could eventually take up all disk spaces and make the whole cluster hard to recover (imagine finding these temp files in thousands of machines). By putting it in /tmp, all these will be automatically removed whenever the machine is restarted, or cleaned up manually by just removing all files in /tmp. That's much easier in terms of Hadoop administration. 2) /tmp seems to be a universally writable mounted point on all unix-based systems. So it should be a safe place to put the temporary files. But I agree that it would a good idea to introduce a new Hive parameter to let the user choose a customizable tmp directory, which should address Edward's point. I think we use File.createTempFile() previously and Yongqiang added another directory structure (parentFile). I think we can use File.createTempFile to create this directory as well (Yongqiang, please correct me if I'm wrong), there there are several questions about the patch: 1) File.createTempFile() will by default create temp files in java.io.tmpdir (/tmp or /var/tmp) according to JDK. I suspect most of the case JVM on Linux will use /tmp as the java.io.tmpdir, so in that caes your change won't achieve your purpose in most cases. 2) the random number generator is not needed if you can use File.createTempFile to create parentFile, because File.createTempFile guarantees its uniqueness for this JVM. The only case you need to prevent is the collision between different JVMs. This is why there is a check for parentFile.mkdir(). It actually acts like a write lock to the directory so that other JVM won't overwrite it. So parentFile.delete() should not be added because it may delete the directory created by another JVM who is still running. > RowContainer uses hard-coded '/tmp/' path for temporary files > - > > Key: HIVE-1326 > URL: https://issues.apache.org/jira/browse/HIVE-1326 > Project: Hadoop Hive > Issue Type: Bug > Environment: Hadoop 0.19.2 with Hive trunk. We're using FreeBSD 7.0, > but that doesn't seem relevant. >Reporter: Michael Klatt > Attachments: rowcontainer.patch > > > In our production hadoop environment, the "/tmp/" is actually pretty small, > and we encountered a problem when a query used the RowContainer class and > filled up the /tmp/ partition. I tracked down the cause to the RowContainer > class putting temporary files in the '/tmp/' path instead of using the > configured Hadoop temporary path. I've attached a patch to fix this. > Here's the traceback: > 2010-04-25 12:05:05,120 INFO > org.apache.hadoop.hive.ql.exec.persistence.RowContainer: RowContainer created > temp file /tmp/hive-rowcontainer-1244151903/RowContainer7816.tmp > 2010-04-25 12:05:06,326 INFO ExecReducer: ExecReducer: processing 1000 > rows: used memory = 385520312 > 2010-04-25 12:05:08,513 INFO ExecReducer: ExecReducer: processing 1100 > rows: used memory = 341780472 > 2010-04-25 12:05:10,697 INFO ExecReducer: ExecReducer: processing 1200 > rows: used memory = 301446768 > 2010-04-25 12:05:12,837 INFO ExecReducer: ExecReducer: processing 1300 > rows: used memory = 399208768 > 2010-04-25 12:05:15,085 INFO ExecReducer: ExecReducer: processing 1400 > rows: used memory = 364507216 > 2010-04-25 12:05:17,260 INFO ExecReducer: ExecReducer: processing 1500 > rows: used memory = 332907280 > 2010-04-25 12:05:19,580 INFO ExecReducer: ExecReducer: processing 1600 > rows: used memory = 298774096 > 2010-04-25 12:05:21,629 INFO ExecReducer: ExecReducer: processing 1700 > rows: used memory = 396505408 > 2010-04-25 12:05:23,830 INFO ExecReducer: ExecReducer: processing 1800 > rows: used memory = 362477288 > 2010-04-25 12:05:25,914 INFO ExecReducer: ExecReducer: processing 1900 > rows: used memory = 327229744 > 2010-04-25 12:05:27,978 INFO ExecReducer: ExecReducer: processing 2000 > rows: used memory = 296051904 > 2010-04-25 12:05:28,155 FATAL ExecReducer: org.apache.hadoop.fs.FSError: > java.io.IOException: No space left on device > at > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:199) > at > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) > at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109) >
[jira] Commented: (HIVE-1326) RowContainer uses hard-coded '/tmp/' path for temporary files
[ https://issues.apache.org/jira/browse/HIVE-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860929#action_12860929 ] Edward Capriolo commented on HIVE-1326: --- I am +1 on the concept. Many times /tmp can be a ramdisk or mounted with some size restrictions. This type of bug an be very painful to track down when it happens. > RowContainer uses hard-coded '/tmp/' path for temporary files > - > > Key: HIVE-1326 > URL: https://issues.apache.org/jira/browse/HIVE-1326 > Project: Hadoop Hive > Issue Type: Bug > Environment: Hadoop 0.19.2 with Hive trunk. We're using FreeBSD 7.0, > but that doesn't seem relevant. >Reporter: Michael Klatt > Attachments: rowcontainer.patch > > > In our production hadoop environment, the "/tmp/" is actually pretty small, > and we encountered a problem when a query used the RowContainer class and > filled up the /tmp/ partition. I tracked down the cause to the RowContainer > class putting temporary files in the '/tmp/' path instead of using the > configured Hadoop temporary path. I've attached a patch to fix this. > Here's the traceback: > 2010-04-25 12:05:05,120 INFO > org.apache.hadoop.hive.ql.exec.persistence.RowContainer: RowContainer created > temp file /tmp/hive-rowcontainer-1244151903/RowContainer7816.tmp > 2010-04-25 12:05:06,326 INFO ExecReducer: ExecReducer: processing 1000 > rows: used memory = 385520312 > 2010-04-25 12:05:08,513 INFO ExecReducer: ExecReducer: processing 1100 > rows: used memory = 341780472 > 2010-04-25 12:05:10,697 INFO ExecReducer: ExecReducer: processing 1200 > rows: used memory = 301446768 > 2010-04-25 12:05:12,837 INFO ExecReducer: ExecReducer: processing 1300 > rows: used memory = 399208768 > 2010-04-25 12:05:15,085 INFO ExecReducer: ExecReducer: processing 1400 > rows: used memory = 364507216 > 2010-04-25 12:05:17,260 INFO ExecReducer: ExecReducer: processing 1500 > rows: used memory = 332907280 > 2010-04-25 12:05:19,580 INFO ExecReducer: ExecReducer: processing 1600 > rows: used memory = 298774096 > 2010-04-25 12:05:21,629 INFO ExecReducer: ExecReducer: processing 1700 > rows: used memory = 396505408 > 2010-04-25 12:05:23,830 INFO ExecReducer: ExecReducer: processing 1800 > rows: used memory = 362477288 > 2010-04-25 12:05:25,914 INFO ExecReducer: ExecReducer: processing 1900 > rows: used memory = 327229744 > 2010-04-25 12:05:27,978 INFO ExecReducer: ExecReducer: processing 2000 > rows: used memory = 296051904 > 2010-04-25 12:05:28,155 FATAL ExecReducer: org.apache.hadoop.fs.FSError: > java.io.IOException: No space left on device > at > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:199) > at > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) > at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109) > at > org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49) > at java.io.DataOutputStream.write(DataOutputStream.java:90) > at > org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.writeChunk(ChecksumFileSystem.java:346) > at > org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:150) > at > org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:132) > at > org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:121) > at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:112) > at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86) > at > org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49) > at java.io.DataOutputStream.write(DataOutputStream.java:90) > at > org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1013) > at > org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977) > at > org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat$1.write(HiveSequenceFileOutputFormat.java:70) > at > org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:343) > at > org.apache.hadoop.hive.ql.exec.persistence.RowContainer.add(RowContainer.java:163) > at > org.apache.hadoop.hive.ql.exec.JoinOperator.processOp(JoinOperator.java:118) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:456) > at > org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:244) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:436) > at org.apache.hadoop.mapred.Child.main(Child.java:158) > Caused by: java.io.IOException: No space left on device > at java.io.FileOutputStream.writeBytes(Native Method) >