nssalian opened a new pull request, #1607: URL: https://github.com/apache/iceberg-go/pull/1607
Adds predicate pushdown for shredded variant fields and rejects Variant as a partition source. Closes [#1016](https://github.com/apache/iceberg-go/issues/1016). ## Changes 1. Extract expressions - `extract(variant, path, type)` referencing a field inside a variant column. RFC-9535 path parse (`variant_path.go`), bind against the shredded schema (`variant_extract.go`), cast to the target type (`variant_cast.go`). Follows Java `Expressions.extract` / `PathUtil.parse` / `VariantExpressionUtil.castTo`. 2. File pruning - inclusive/strict metrics evaluators use a shredded field's stored bounds for an extract term (`table/evaluators.go`); the strict path never prunes on extract. Bounds are split via arrow-go `Metadata.SizeBytes()` (`table/internal/variant_bounds.go`). Follows Java `InclusiveMetricsEvaluator` /`StrictMetricsEvaluator`. 3. Residual filtering (columnar) - surviving extract terms become synthetic derived columns evaluated through the existing arrow-go `compute` record-filter path, not row-by-row (`scanTranslator` in `visitors.go`, `table/variant_residual.go`,`table/arrow_scanner.go`). 4. Reject Variant partition source - `addSpecFieldInternal` (`partitions.go`) rejects a Variant source for any transform. Builds on [#1206](https://github.com/apache/iceberg-go/pull/1206) (which added Variant to `IdentityTransform`'s deny-list); this path does not consult `CanTransform`, so it closes the gap for sources added via `AddPartitionFieldBySourceID`. ## Testing - Unit tests for path parse, cast, extract binding, residual eval, metrics pruning, and partition rejection; end-to-end shredded-write + pushdown test (`table/variant_shredded_write_test.go`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
