kokila-19 commented on code in PR #6138:
URL: https://github.com/apache/hive/pull/6138#discussion_r2452028687
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -929,9 +934,64 @@ public DynamicPartitionCtx createDPContext(
addCustomSortExpr(table, hmsTable, writeOperation, dpCtx,
getSortTransformSpec(table));
}
+ // Even if table has no explicit sort order, honor z-order if configured
+ Map<String, String> props = table.properties();
+ if ("ZORDER".equalsIgnoreCase(props.getOrDefault(SORT_ORDER, ""))) {
+ createZOrderCustomSort(props, dpCtx, table, hmsTable, writeOperation);
+ }
+
return dpCtx;
}
+ /**
+ * Adds a custom sort expression to the DynamicPartitionCtx that performs
local Z-ordering on write.
+ *
+ * Behavior:
+ * - Reads Z-order properties from 'sort.order' and 'sort.columns'
(comma-separated).
+ * - Resolves the referenced columns to their positions in the physical row
(taking into account
+ * ACID virtual columns offset for overwrite/update operations).
+ * - Configures a single ASC sort key with NULLS FIRST and injects a custom
key expression for
+ * Z-order
+ */
+ private void createZOrderCustomSort(Map<String, String> props,
DynamicPartitionCtx dpCtx, Table table,
+ org.apache.hadoop.hive.ql.metadata.Table hmsTable, Operation
writeOperation) {
+ String colsProp = props.get(SORT_COLUMNS);
+ if (StringUtils.isNotBlank(colsProp)) {
+ List<String> zCols = Arrays.stream(colsProp.split(",")).map(String::trim)
+ .filter(s -> !s.isEmpty()).collect(Collectors.toList());
+
+ Map<String, Integer> fieldOrderMap = Maps.newHashMap();
Review Comment:
Using `new HashMap` will trigger checkstyle error in iceberg module.
```
<module name="RegexpSingleline">
<property name="format" value="new HashMap<>\(.*\)"/>
<property name="message"
value="Prefer using Maps.newHashMap instead."/>
```
`[ERROR]
/Users/kokila/cdp/upstream/up_hive/hive/iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:967:
Prefer using Maps.newHashMap instead. [RegexpSingleline]`
Map<String, Integer> fieldOrderMap =
Maps.newHashMapWithExpectedSize(fields.size()); - This can be used instead to
avoid checkstyle error and also with specific size
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
