[jira] Commented: (HIVE-1326) RowContainer uses hard-coded '/tmp/' path for temporary files

2010-04-26 Thread Ning Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861188#action_12861188
 ] 

Ning Zhang commented on HIVE-1326:
--

I guess this is due to https://issues.apache.org/jira/browse/HADOOP-2735 which 
set java.io.tmpdir to be dependent on mapred.tmp.dir. In that case the change 
of using createTempFile looks fine with me. Also can you remove the 
parentFile.delete() in your patch? If we set java.io.tmpdir to work/tmp then we 
don't have file name conflicts, But if the user choose to use shared /tmp then 
it may remove files created by other tasks. 

Yongqiang, any further comments?

> RowContainer uses hard-coded '/tmp/' path for temporary files
> -
>
> Key: HIVE-1326
> URL: https://issues.apache.org/jira/browse/HIVE-1326
> Project: Hadoop Hive
>  Issue Type: Bug
> Environment: Hadoop 0.19.2 with Hive trunk.  We're using FreeBSD 7.0, 
> but that doesn't seem relevant.
>Reporter: Michael Klatt
> Attachments: rowcontainer.patch
>
>
> In our production hadoop environment, the "/tmp/" is actually pretty small, 
> and we encountered a problem when a query used the RowContainer class and 
> filled up the /tmp/ partition.  I tracked down the cause to the RowContainer 
> class putting temporary files in the '/tmp/' path instead of using the 
> configured Hadoop temporary path.  I've attached a patch to fix this.
> Here's the traceback:
> 2010-04-25 12:05:05,120 INFO 
> org.apache.hadoop.hive.ql.exec.persistence.RowContainer: RowContainer created 
> temp file /tmp/hive-rowcontainer-1244151903/RowContainer7816.tmp
> 2010-04-25 12:05:06,326 INFO ExecReducer: ExecReducer: processing 1000 
> rows: used memory = 385520312
> 2010-04-25 12:05:08,513 INFO ExecReducer: ExecReducer: processing 1100 
> rows: used memory = 341780472
> 2010-04-25 12:05:10,697 INFO ExecReducer: ExecReducer: processing 1200 
> rows: used memory = 301446768
> 2010-04-25 12:05:12,837 INFO ExecReducer: ExecReducer: processing 1300 
> rows: used memory = 399208768
> 2010-04-25 12:05:15,085 INFO ExecReducer: ExecReducer: processing 1400 
> rows: used memory = 364507216
> 2010-04-25 12:05:17,260 INFO ExecReducer: ExecReducer: processing 1500 
> rows: used memory = 332907280
> 2010-04-25 12:05:19,580 INFO ExecReducer: ExecReducer: processing 1600 
> rows: used memory = 298774096
> 2010-04-25 12:05:21,629 INFO ExecReducer: ExecReducer: processing 1700 
> rows: used memory = 396505408
> 2010-04-25 12:05:23,830 INFO ExecReducer: ExecReducer: processing 1800 
> rows: used memory = 362477288
> 2010-04-25 12:05:25,914 INFO ExecReducer: ExecReducer: processing 1900 
> rows: used memory = 327229744
> 2010-04-25 12:05:27,978 INFO ExecReducer: ExecReducer: processing 2000 
> rows: used memory = 296051904
> 2010-04-25 12:05:28,155 FATAL ExecReducer: org.apache.hadoop.fs.FSError: 
> java.io.IOException: No space left on device
>   at 
> org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:199)
>   at 
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>   at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
>   at 
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49)
>   at java.io.DataOutputStream.write(DataOutputStream.java:90)
>   at 
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.writeChunk(ChecksumFileSystem.java:346)
>   at 
> org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:150)
>   at 
> org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:132)
>   at 
> org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:121)
>   at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:112)
>   at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86)
>   at 
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49)
>   at java.io.DataOutputStream.write(DataOutputStream.java:90)
>   at 
> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1013)
>   at 
> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
>   at 
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat$1.write(HiveSequenceFileOutputFormat.java:70)
>   at 
> org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:343)
>   at 
> org.apache.hadoop.hive.ql.exec.persistence.RowContainer.add(RowContainer.java:163)
>   at 
> org.apache.hadoop.hive.ql.exec.JoinOperator.processOp(JoinOperator.java:118)
>   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:456)
>   at 
> org.apache.hadoop.hive.ql.exec.ExecReducer.r

[jira] Updated: (HIVE-1325) dynamic partition insert should throw an exception if the number of target table columns + dynamic partition columns does not equal to the number of select columns

2010-04-26 Thread Namit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-1325:
-

  Status: Resolved  (was: Patch Available)
Hadoop Flags: [Reviewed]
  Resolution: Fixed

Committed. Thanks Ning

> dynamic partition insert should throw an exception if the number of target 
> table columns + dynamic partition columns does not equal to the number of 
> select columns
> ---
>
> Key: HIVE-1325
> URL: https://issues.apache.org/jira/browse/HIVE-1325
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: Ning Zhang
>Assignee: Ning Zhang
> Fix For: 0.6.0
>
> Attachments: HIVE-1325.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (HIVE-1321) bugs with temp directories, trailing blank fields in HBase bulk load

2010-04-26 Thread Namit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-1321:
-

  Status: Resolved  (was: Patch Available)
Hadoop Flags: [Reviewed]
  Resolution: Fixed

Committed. Thanks John

> bugs with temp directories, trailing blank fields in HBase bulk load
> 
>
> Key: HIVE-1321
> URL: https://issues.apache.org/jira/browse/HIVE-1321
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Affects Versions: 0.6.0
>Reporter: John Sichi
>Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1321.1.patch
>
>
> HIVE-1295 had two bugs discovered during testing with production data:
> (1) extra directories may be present in the output directory depending on how 
> the cluster is configured; we need to walk down these to find the column 
> family directory
> (2) if a record ends with fields which are blank strings, the text format 
> omits the corresponding Control-A delimiters, so we need to fill in blanks 
> for these fields (instead of throwing ArrayIndexOutOfBoundsException)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (HIVE-1325) dynamic partition insert should throw an exception if the number of target table columns + dynamic partition columns does not equal to the number of select columns

2010-04-26 Thread Namit Jain (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861127#action_12861127
 ] 

Namit Jain commented on HIVE-1325:
--

+1

looks good - will commit if the tests pass

> dynamic partition insert should throw an exception if the number of target 
> table columns + dynamic partition columns does not equal to the number of 
> select columns
> ---
>
> Key: HIVE-1325
> URL: https://issues.apache.org/jira/browse/HIVE-1325
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: Ning Zhang
>Assignee: Ning Zhang
> Fix For: 0.6.0
>
> Attachments: HIVE-1325.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (HIVE-1325) dynamic partition insert should throw an exception if the number of target table columns + dynamic partition columns does not equal to the number of select columns

2010-04-26 Thread Namit Jain (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861125#action_12861125
 ] 

Namit Jain commented on HIVE-1325:
--

I will take a look

> dynamic partition insert should throw an exception if the number of target 
> table columns + dynamic partition columns does not equal to the number of 
> select columns
> ---
>
> Key: HIVE-1325
> URL: https://issues.apache.org/jira/browse/HIVE-1325
> Project: Hadoop Hive
>  Issue Type: Bug
>Reporter: Ning Zhang
>Assignee: Ning Zhang
> Fix For: 0.6.0
>
> Attachments: HIVE-1325.2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (HIVE-1326) RowContainer uses hard-coded '/tmp/' path for temporary files

2010-04-26 Thread Michael Klatt (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861080#action_12861080
 ] 

Michael Klatt commented on HIVE-1326:
-


In my brief experiments, I found that File.createTempFile() used the work/tmp 
path for the individual task (in Hadoop).  I suspect that the java.io.tmpdir is 
being set by the TaskTracker.  This has the advantage of letting Hadoop do the 
temporary file cleanup when the task cleanup happens.  I think it would be 
great if this was the default behavior, but admins could override that with a 
hive specific parameter.

I don't have any attachment to the actual implementation.  I can spend more 
time to produce a different patch if needed, but I suspected someone would want 
to implement it differently.

> RowContainer uses hard-coded '/tmp/' path for temporary files
> -
>
> Key: HIVE-1326
> URL: https://issues.apache.org/jira/browse/HIVE-1326
> Project: Hadoop Hive
>  Issue Type: Bug
> Environment: Hadoop 0.19.2 with Hive trunk.  We're using FreeBSD 7.0, 
> but that doesn't seem relevant.
>Reporter: Michael Klatt
> Attachments: rowcontainer.patch
>
>
> In our production hadoop environment, the "/tmp/" is actually pretty small, 
> and we encountered a problem when a query used the RowContainer class and 
> filled up the /tmp/ partition.  I tracked down the cause to the RowContainer 
> class putting temporary files in the '/tmp/' path instead of using the 
> configured Hadoop temporary path.  I've attached a patch to fix this.
> Here's the traceback:
> 2010-04-25 12:05:05,120 INFO 
> org.apache.hadoop.hive.ql.exec.persistence.RowContainer: RowContainer created 
> temp file /tmp/hive-rowcontainer-1244151903/RowContainer7816.tmp
> 2010-04-25 12:05:06,326 INFO ExecReducer: ExecReducer: processing 1000 
> rows: used memory = 385520312
> 2010-04-25 12:05:08,513 INFO ExecReducer: ExecReducer: processing 1100 
> rows: used memory = 341780472
> 2010-04-25 12:05:10,697 INFO ExecReducer: ExecReducer: processing 1200 
> rows: used memory = 301446768
> 2010-04-25 12:05:12,837 INFO ExecReducer: ExecReducer: processing 1300 
> rows: used memory = 399208768
> 2010-04-25 12:05:15,085 INFO ExecReducer: ExecReducer: processing 1400 
> rows: used memory = 364507216
> 2010-04-25 12:05:17,260 INFO ExecReducer: ExecReducer: processing 1500 
> rows: used memory = 332907280
> 2010-04-25 12:05:19,580 INFO ExecReducer: ExecReducer: processing 1600 
> rows: used memory = 298774096
> 2010-04-25 12:05:21,629 INFO ExecReducer: ExecReducer: processing 1700 
> rows: used memory = 396505408
> 2010-04-25 12:05:23,830 INFO ExecReducer: ExecReducer: processing 1800 
> rows: used memory = 362477288
> 2010-04-25 12:05:25,914 INFO ExecReducer: ExecReducer: processing 1900 
> rows: used memory = 327229744
> 2010-04-25 12:05:27,978 INFO ExecReducer: ExecReducer: processing 2000 
> rows: used memory = 296051904
> 2010-04-25 12:05:28,155 FATAL ExecReducer: org.apache.hadoop.fs.FSError: 
> java.io.IOException: No space left on device
>   at 
> org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:199)
>   at 
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>   at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
>   at 
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49)
>   at java.io.DataOutputStream.write(DataOutputStream.java:90)
>   at 
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.writeChunk(ChecksumFileSystem.java:346)
>   at 
> org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:150)
>   at 
> org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:132)
>   at 
> org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:121)
>   at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:112)
>   at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86)
>   at 
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49)
>   at java.io.DataOutputStream.write(DataOutputStream.java:90)
>   at 
> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1013)
>   at 
> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
>   at 
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat$1.write(HiveSequenceFileOutputFormat.java:70)
>   at 
> org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:343)
>   at 
> org.apache.hadoop.hive.ql.exec.persistence.RowContainer.add(RowContainer.java:163)
>   at 
> org.apache.hadoop.hive.ql.exec.JoinOperator.processOp(JoinOperator

[jira] Commented: (HIVE-1326) RowContainer uses hard-coded '/tmp/' path for temporary files

2010-04-26 Thread Ning Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861054#action_12861054
 ] 

Ning Zhang commented on HIVE-1326:
--

The general idea of why we put temp files and directories under /tmp is two 
fold:
 1) it allows easy clean up in case of any unexpected interruptions to the JVM. 
In the regular exits, these temp files/directories should be removed by JVM by 
deleteOnExit(), but if the JVM was killed/interrupted unexpected, these temp 
files could eventually take up all disk spaces and make the whole cluster hard 
to recover (imagine finding these temp files in thousands of machines). By 
putting it in /tmp, all these will be automatically removed whenever the 
machine is restarted, or cleaned up manually by just removing all files in 
/tmp. That's much easier in terms of Hadoop administration. 
 2) /tmp seems to be a universally writable mounted point on all unix-based 
systems. So it should be a safe place to put the temporary files. But I agree 
that it would a good idea to introduce a new Hive parameter to let the user 
choose a customizable tmp directory, which should address Edward's point. 
 
I think we use File.createTempFile() previously and Yongqiang added another 
directory structure (parentFile). I think we can use File.createTempFile to 
create this directory as well (Yongqiang, please correct me if I'm wrong), 
there there are several questions about the patch:

1) File.createTempFile() will by default create temp files in java.io.tmpdir 
(/tmp or /var/tmp) according to JDK. I suspect most of the case JVM on Linux 
will use /tmp as the java.io.tmpdir, so in that caes your change won't achieve 
your purpose in most cases. 
2) the random number generator is not needed if you can use File.createTempFile 
to create parentFile, because File.createTempFile guarantees its uniqueness for 
this JVM. The only case you need to prevent is the collision between different 
JVMs. This is why there is a check for parentFile.mkdir(). It actually acts 
like a write lock to the directory so that other JVM won't overwrite it. So 
parentFile.delete() should not be added because it may delete the directory 
created by another JVM who is still running.


> RowContainer uses hard-coded '/tmp/' path for temporary files
> -
>
> Key: HIVE-1326
> URL: https://issues.apache.org/jira/browse/HIVE-1326
> Project: Hadoop Hive
>  Issue Type: Bug
> Environment: Hadoop 0.19.2 with Hive trunk.  We're using FreeBSD 7.0, 
> but that doesn't seem relevant.
>Reporter: Michael Klatt
> Attachments: rowcontainer.patch
>
>
> In our production hadoop environment, the "/tmp/" is actually pretty small, 
> and we encountered a problem when a query used the RowContainer class and 
> filled up the /tmp/ partition.  I tracked down the cause to the RowContainer 
> class putting temporary files in the '/tmp/' path instead of using the 
> configured Hadoop temporary path.  I've attached a patch to fix this.
> Here's the traceback:
> 2010-04-25 12:05:05,120 INFO 
> org.apache.hadoop.hive.ql.exec.persistence.RowContainer: RowContainer created 
> temp file /tmp/hive-rowcontainer-1244151903/RowContainer7816.tmp
> 2010-04-25 12:05:06,326 INFO ExecReducer: ExecReducer: processing 1000 
> rows: used memory = 385520312
> 2010-04-25 12:05:08,513 INFO ExecReducer: ExecReducer: processing 1100 
> rows: used memory = 341780472
> 2010-04-25 12:05:10,697 INFO ExecReducer: ExecReducer: processing 1200 
> rows: used memory = 301446768
> 2010-04-25 12:05:12,837 INFO ExecReducer: ExecReducer: processing 1300 
> rows: used memory = 399208768
> 2010-04-25 12:05:15,085 INFO ExecReducer: ExecReducer: processing 1400 
> rows: used memory = 364507216
> 2010-04-25 12:05:17,260 INFO ExecReducer: ExecReducer: processing 1500 
> rows: used memory = 332907280
> 2010-04-25 12:05:19,580 INFO ExecReducer: ExecReducer: processing 1600 
> rows: used memory = 298774096
> 2010-04-25 12:05:21,629 INFO ExecReducer: ExecReducer: processing 1700 
> rows: used memory = 396505408
> 2010-04-25 12:05:23,830 INFO ExecReducer: ExecReducer: processing 1800 
> rows: used memory = 362477288
> 2010-04-25 12:05:25,914 INFO ExecReducer: ExecReducer: processing 1900 
> rows: used memory = 327229744
> 2010-04-25 12:05:27,978 INFO ExecReducer: ExecReducer: processing 2000 
> rows: used memory = 296051904
> 2010-04-25 12:05:28,155 FATAL ExecReducer: org.apache.hadoop.fs.FSError: 
> java.io.IOException: No space left on device
>   at 
> org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:199)
>   at 
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>   at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
>

[jira] Commented: (HIVE-1326) RowContainer uses hard-coded '/tmp/' path for temporary files

2010-04-26 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860929#action_12860929
 ] 

Edward Capriolo commented on HIVE-1326:
---

I am +1 on the concept.   Many times /tmp can be a ramdisk or mounted with some 
size restrictions. This type of bug an be very painful to track down when it 
happens.

> RowContainer uses hard-coded '/tmp/' path for temporary files
> -
>
> Key: HIVE-1326
> URL: https://issues.apache.org/jira/browse/HIVE-1326
> Project: Hadoop Hive
>  Issue Type: Bug
> Environment: Hadoop 0.19.2 with Hive trunk.  We're using FreeBSD 7.0, 
> but that doesn't seem relevant.
>Reporter: Michael Klatt
> Attachments: rowcontainer.patch
>
>
> In our production hadoop environment, the "/tmp/" is actually pretty small, 
> and we encountered a problem when a query used the RowContainer class and 
> filled up the /tmp/ partition.  I tracked down the cause to the RowContainer 
> class putting temporary files in the '/tmp/' path instead of using the 
> configured Hadoop temporary path.  I've attached a patch to fix this.
> Here's the traceback:
> 2010-04-25 12:05:05,120 INFO 
> org.apache.hadoop.hive.ql.exec.persistence.RowContainer: RowContainer created 
> temp file /tmp/hive-rowcontainer-1244151903/RowContainer7816.tmp
> 2010-04-25 12:05:06,326 INFO ExecReducer: ExecReducer: processing 1000 
> rows: used memory = 385520312
> 2010-04-25 12:05:08,513 INFO ExecReducer: ExecReducer: processing 1100 
> rows: used memory = 341780472
> 2010-04-25 12:05:10,697 INFO ExecReducer: ExecReducer: processing 1200 
> rows: used memory = 301446768
> 2010-04-25 12:05:12,837 INFO ExecReducer: ExecReducer: processing 1300 
> rows: used memory = 399208768
> 2010-04-25 12:05:15,085 INFO ExecReducer: ExecReducer: processing 1400 
> rows: used memory = 364507216
> 2010-04-25 12:05:17,260 INFO ExecReducer: ExecReducer: processing 1500 
> rows: used memory = 332907280
> 2010-04-25 12:05:19,580 INFO ExecReducer: ExecReducer: processing 1600 
> rows: used memory = 298774096
> 2010-04-25 12:05:21,629 INFO ExecReducer: ExecReducer: processing 1700 
> rows: used memory = 396505408
> 2010-04-25 12:05:23,830 INFO ExecReducer: ExecReducer: processing 1800 
> rows: used memory = 362477288
> 2010-04-25 12:05:25,914 INFO ExecReducer: ExecReducer: processing 1900 
> rows: used memory = 327229744
> 2010-04-25 12:05:27,978 INFO ExecReducer: ExecReducer: processing 2000 
> rows: used memory = 296051904
> 2010-04-25 12:05:28,155 FATAL ExecReducer: org.apache.hadoop.fs.FSError: 
> java.io.IOException: No space left on device
>   at 
> org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:199)
>   at 
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>   at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
>   at 
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49)
>   at java.io.DataOutputStream.write(DataOutputStream.java:90)
>   at 
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.writeChunk(ChecksumFileSystem.java:346)
>   at 
> org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:150)
>   at 
> org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:132)
>   at 
> org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:121)
>   at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:112)
>   at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86)
>   at 
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49)
>   at java.io.DataOutputStream.write(DataOutputStream.java:90)
>   at 
> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1013)
>   at 
> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
>   at 
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat$1.write(HiveSequenceFileOutputFormat.java:70)
>   at 
> org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:343)
>   at 
> org.apache.hadoop.hive.ql.exec.persistence.RowContainer.add(RowContainer.java:163)
>   at 
> org.apache.hadoop.hive.ql.exec.JoinOperator.processOp(JoinOperator.java:118)
>   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:456)
>   at 
> org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:244)
>   at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:436)
>   at org.apache.hadoop.mapred.Child.main(Child.java:158)
> Caused by: java.io.IOException: No space left on device
>   at java.io.FileOutputStream.writeBytes(Native Method)
>