pvary commented on a change in pull request #1327:
URL: https://github.com/apache/hive/pull/1327#discussion_r463491598
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
##########
@@ -1063,7 +1076,11 @@ public void process(Object row, int tag) throws
HiveException {
// RecordUpdater expects to get the actual row, not a serialized version
of it. Thus we
// pass the row rather than recordValue.
if (conf.getWriteType() == AcidUtils.Operation.NOT_ACID ||
conf.isMmTable() || conf.isCompactionTable()) {
- rowOutWriters[findWriterOffset(row)].write(recordValue);
+ writerOffset = bucketId;
+ if (!conf.isCompactionTable()) {
+ writerOffset = findWriterOffset(row);
+ }
+ rowOutWriters[writerOffset].write(recordValue);
Review comment:
Are the results in order? Can we close writers for old buckets? That
could save memory
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
