anew opened a new pull request, #57245: URL: https://github.com/apache/spark/pull/57245
### What changes were proposed in this pull request? `GraphValidations.validateUserSpecifiedSchemas` looked up a table by the incoming flow's own identifier (`table.get(f.identifier)`). That only matches when the flow is an implicit/default flow whose identifier equals its destination table's. For a named flow (e.g. `CREATE FLOW <name> AS AUTO CDC INTO <target>`, or any explicitly-named flow) the lookup returned `None`, so the table's declared schema was never validated against the inferred schema. This changes the lookup to key on `f.destinationIdentifier` (with `.distinct`, since multiple flows can share a destination), matching the sibling `validateFlowStreamingness` validation. ### Why are the changes needed? An incompatible user-declared schema on a named-flow table went undetected at graph-validation time and surfaced only as a confusing mid-stream runtime failure at materialization. ### Does this introduce _any_ user-facing change? Yes: an incompatible user-specified schema on a table fed by a named flow now fails with `USER_SPECIFIED_AND_INFERRED_SCHEMA_NOT_COMPATIBLE` at validation time, as it already did for the implicit-flow form. ### How was this patch tested? New `UserSpecifiedSchemaValidationSuite` asserts the error is raised for both the implicit- and named-flow forms; confirmed the named-flow case fails without the fix. Ran ConnectValid/Invalid, SqlPipeline, and AutoCdc pipeline suites (130 tests) green. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 4.8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
