dejankrak-db commented on PR #59694: URL: https://github.com/apache/spark/pull/59694#issuecomment-6343185334
@cloud-fan, thanks, you were right on both shapes. I ran them on Spark Connect, in strict and non-strict resolution, against the merge base and this PR: shape 1 resolved `W["b.k"]` to `S2`'s own `k` (different rows, no error), and shape 2 failed with `AMBIGUOUS_COLUMN_REFERENCE` where the merge base returns `[10, 20]`. In 4c90759b989 a node now hides the attributes it introduces in its own `metadataOutput` (those not in any child's `metadataOutput`), even when `AddMetadataColumns` has added them to its output. That covers the widened USING join wrapper and relations that expose metadata columns. A re-selecting Project (`sel`) or a USING join wrapper above it only inherits the key, so it stays visible there. With this, both shapes match the merge base, and this PR's test and the SPARK-56632 test still pass. I added both shapes to the Connect parity tests; in classic mode they are rejected by the existing self-join ambiguity check, so they are Connect-only. I also updated the PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
