[GitHub] [spark] grundprinzip commented on pull request #41933: [SPARK-44370][CONNECT] Migrate Buf remote generation alpha to remote plugins

2023-07-11 Thread via GitHub
grundprinzip commented on PR #41933: URL: https://github.com/apache/spark/pull/41933#issuecomment-1631139062 My suggestion would be to merge the proposed version with v21.7 and assuming that the integration test pass should be still compatible. This should unblock any new proto chang

[GitHub] [spark] grundprinzip commented on pull request #41933: [SPARK-44370][CONNECT] Migrate Buf remote generation alpha to remote plugins

2023-07-11 Thread via GitHub
grundprinzip commented on PR #41933: URL: https://github.com/apache/spark/pull/41933#issuecomment-1631004094 TBH I don't know how exactly to fix this problem. The buf remote plugins don't offer the protobuf version we need, so maybe the fix is to switch from remote to local plugins. -- T

[GitHub] [spark] grundprinzip commented on pull request #41933: [SPARK-44370][CONNECT] Migrate Buf remote generation alpha to remote plugins

2023-07-11 Thread via GitHub
grundprinzip commented on PR #41933: URL: https://github.com/apache/spark/pull/41933#issuecomment-1630653264 I think using the 21.7 version might not be very risky, but we should make sure that we stay consistent in our protobuf usage and we need to check if this should apply as well to upg

[GitHub] [spark] grundprinzip commented on pull request #41933: [SPARK-44370][CONNECT] Migrate Buf remote generation alpha to remote plugins

2023-07-11 Thread via GitHub
grundprinzip commented on PR #41933: URL: https://github.com/apache/spark/pull/41933#issuecomment-1630651148 Do we know what the changes in the generated code are? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [spark] grundprinzip commented on pull request #41933: [SPARK-44370][CONNECT] Migrate Buf remote generation alpha to remote plugins

2023-07-11 Thread via GitHub
grundprinzip commented on PR #41933: URL: https://github.com/apache/spark/pull/41933#issuecomment-1630637888 We have to be careful that the remotely generated protobuf code matches our protobuf version locally because otherwise we will run into compatibility issues. Protobuf had some change