deniskuzZ commented on code in PR #6789:
URL: https://github.com/apache/hive/pull/6789#discussion_r4313652650
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergInputFormat.java:
##########
@@ -72,31 +72,36 @@
*/
public class IcebergInputFormat<T> extends InputFormat<Void, T> {
- private static TableScan createTableScan(Table table, Configuration conf) {
+ private static TableScan createTableScan(Table table, Configuration conf,
Table serverSidePinTable) {
TableScan scan = table.newScan();
- long snapshotId = -1;
- try {
- snapshotId = conf.getLong(InputFormatConfig.SNAPSHOT_ID, -1);
- } catch (NumberFormatException e) {
- String version = conf.get(InputFormatConfig.SNAPSHOT_ID);
- SnapshotRef ref = table.refs().get(version);
- if (ref == null) {
- throw new RuntimeException("Cannot find matching snapshot ID or
reference name for version " + version);
+ if (HiveTableUtil.hasExplicitScanSnapshotBinding(conf)) {
+ long snapshotId = -1;
+ try {
+ snapshotId = conf.getLong(InputFormatConfig.SNAPSHOT_ID, -1);
+ } catch (NumberFormatException e) {
+ String version = conf.get(InputFormatConfig.SNAPSHOT_ID);
+ Table refsTable = serverSidePinTable != null ? serverSidePinTable :
table;
+ SnapshotRef ref = refsTable.refs().get(version);
+ if (ref == null) {
+ throw new RuntimeException("Cannot find matching snapshot ID or
reference name for version " + version);
+ }
+ snapshotId = ref.snapshotId();
+ }
+ String refName = conf.get(InputFormatConfig.OUTPUT_TABLE_SNAPSHOT_REF);
+ if (StringUtils.isNotEmpty(refName)) {
Review Comment:
For `t.branch_x`, `OUTPUT_TABLE_SNAPSHOT_REF` is set,
`hasExplicitScanSnapshotBinding` returns true, and `scan.useRef(...)` resolves
the branch on the live `RESTTable` in the AM. Could we resolve that ref against
the serialized table too (as done for `SNAPSHOT_ID` names) and add an IT that
advances the branch after compile?
````
if (StringUtils.isNotEmpty(refName)) {
String branch = HiveUtils.getTableSnapshotRef(refName);
if (serverSidePinTable != null) {
SnapshotRef ref = serverSidePinTable.refs().get(branch);
if (ref == null) {
throw new RuntimeException("Cannot find matching reference name " +
branch);
}
scan = scan.useSnapshot(ref.snapshotId()); // compile-time branch head
} else {
scan = scan.useRef(branch); // client-side path
unchanged
}
}
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
