difin commented on code in PR #6789:
URL: https://github.com/apache/hive/pull/6789#discussion_r4331622712
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergInputFormat.java:
##########
@@ -72,31 +72,36 @@
*/
public class IcebergInputFormat<T> extends InputFormat<Void, T> {
- private static TableScan createTableScan(Table table, Configuration conf) {
+ private static TableScan createTableScan(Table table, Configuration conf,
Table serverSidePinTable) {
TableScan scan = table.newScan();
- long snapshotId = -1;
- try {
- snapshotId = conf.getLong(InputFormatConfig.SNAPSHOT_ID, -1);
- } catch (NumberFormatException e) {
- String version = conf.get(InputFormatConfig.SNAPSHOT_ID);
- SnapshotRef ref = table.refs().get(version);
- if (ref == null) {
- throw new RuntimeException("Cannot find matching snapshot ID or
reference name for version " + version);
+ if (HiveTableUtil.hasExplicitScanSnapshotBinding(conf)) {
+ long snapshotId = -1;
+ try {
+ snapshotId = conf.getLong(InputFormatConfig.SNAPSHOT_ID, -1);
+ } catch (NumberFormatException e) {
+ String version = conf.get(InputFormatConfig.SNAPSHOT_ID);
+ Table refsTable = serverSidePinTable != null ? serverSidePinTable :
table;
+ SnapshotRef ref = refsTable.refs().get(version);
+ if (ref == null) {
+ throw new RuntimeException("Cannot find matching snapshot ID or
reference name for version " + version);
+ }
+ snapshotId = ref.snapshotId();
+ }
+ String refName = conf.get(InputFormatConfig.OUTPUT_TABLE_SNAPSHOT_REF);
+ if (StringUtils.isNotEmpty(refName)) {
Review Comment:
Done - on the server-side path the ref is now resolved against the
serialized table and pinned with `useSnapshot`. One exception: if a branch head
was written with an older schema, I keep `useRef`. Iceberg reads branches with
the table schema, and `useSnapshot` would switch to the snapshot schema and
drop columns added since. Added ITs for both cases: a branch advanced after
compile, and a branch with an older schema.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
