deniskuzZ commented on code in PR #6834:
URL: https://github.com/apache/hive/pull/6834#discussion_r4334978366
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/mapred/MapredIcebergInputFormat.java:
##########
@@ -116,9 +113,13 @@ public Container<T> createValue() {
return new Container<>();
}
+ /**
+ * A split is a group of file scan tasks, so no position within it maps to
a file offset. -1 keeps
+ * HiveContextAwareRecordReader from taking every row for the start of a
file.
+ */
@Override
public long getPos() throws IOException {
- return (long) (splitLength * getProgress());
+ return -1;
Review Comment:
An IcebergSplit is a group of FileScanTasks, so no position inside it maps
to an offset in one file, and the old value (split length × progress) was
invented.
-1 keeps HiveContextAwareRecordReader from taking every row for the start of
a file. BLOCK__OFFSET__INSIDE__FILE is now -1 for Iceberg row-mode reads, and
ROW__POSITION gives the row's position in its data file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
