Yicong-Huang commented on PR #55670: URL: https://github.com/apache/spark/pull/55670#issuecomment-4374394841
> I saw this kind backporting before, but this kind of approach looks still suspicious to me because this only for next releases. Technically, there is no way for the released Apache Spark distributions like 4.0.0 or 4.1.0, isn't it? We may ignore once or twice, but this doesn't look sustainable to me at all. Yeah I understand your concern! The series of change and back ports are all related to "error messages" in PySpark. The effort is to generalize the effort message to not be UDF eval type dependent. We believe this is not breaking backward compatibility requirement as the error type is not changing. Users with old client like 4.0.0 and 4.1.0 will see a different error message, but the handling logic will still work and it is rare that they need change. Our CI on 4.0 and 4.1 is designed too tight, on the other hand. It is doing exact string matching of error messages. These backports are needed mainly for fixing our CIs so that it can continue detect real backward incompatibilities in the future. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
