gaogaotiantian commented on PR #55201: URL: https://github.com/apache/spark/pull/55201#issuecomment-4194911340
My suggestion would be - keep all the worker related changes in this PR and remove the regression test. Then have a new PR for the regression test in master (only the simple worker for UDF, python data source is already tested against simple worker now). If we do want to keep the regression test for all the previous branches, just keep the simplest worker one. If we decide to keep the test in this PR (or even if we move it to another PR), let's just remove the comment - because it's actually something pretty generic that we should test - basically just simple worker. We probably will add more tests into the class in the future and we don't want to bind it to a specific spark issue. Actually I was wondering if we can just reuse the `WorkerTests` and set a config to it just like what we did for python data source simple workers. Anyway, I think the fix itself is good to backport, let's first decide whether we want to backport the regression tests to branch 3.5 - 4.1 first @HyukjinKwon , then we can decide the format. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
