szehon-ho commented on PR #58370:
URL: https://github.com/apache/spark/pull/58370#issuecomment-5459188804
Thanks for the detailed follow-up. Addressed the review in 1983e1593f3 and
24ac1aaac30:
- changed `V2ExpressionUtils.resolveAttributeRefs` to resolve
`NamedExpression`s;
- simplified nested row-level aggregation and left projection insertion to
`PullOutGroupingExpressions`;
- split fully-pushed attribute validation from interface validation;
- added `DATA_SOURCE_INVALID_RUNTIME_FILTER_ATTRIBUTE` (SQLSTATE `KD000`)
with the attribute, scan class, and read schema, while preserving the original
resolution error as the cause;
- made the Catalyst runtime-filter fixture consistently match
`fieldNames.mkString(".")`;
- changed the optional expected filter references to `Option[Seq[String]]`;
- added nested MERGE coverage that runs for both group- and delta-based
row-level operations; and
- updated the PR description to distinguish restored scan compatibility from
the new nested row-level capability.
Validation:
- `DataSourceV2CatalystRuntimeFilterSuite`,
`GroupBasedRowLevelOperationCatalystRuntimeFilterSuite`, and
`DeltaBasedRowLevelOperationCatalystRuntimeFilterSuite`: 29/29 passed
- focused V1/V2 nested partition source tests: 2/2 passed
- `SparkThrowableSuite`: 37/37 passed, including error-catalog formatting
and SQLSTATE invariants
- Catalyst and SQL scalastyle checks passed
The error-catalog golden file was regenerated with the prescribed
`SparkThrowableSuite` command; no SQL query golden outputs apply to these
synthetic scan fixtures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
