Hi all, In SPARK-42452 (apache/spark#41153 [1]), I’m trying to remove protobuf 2.5.0 from the Spark dependencies.
Spark does not use protobuf 2.5.0 directly, instead, it comes from other dependencies, with the following changes, now, Spark does not require protobuf 2.5.0. - SPARK-40323 upgraded ORC 1.8.0, which moved from protobuf 2.5.0 to a shaded protobuf 3 - SPARK-33212 switched from Hadoop vanilla client to Hadoop shaded client, also removed the protobuf 2 dependency. SPARK-42452 removed the support for Hadoop 2. - SPARK-14421 shaded and relocated protobuf 2.6.1, which is required by the kinesis client, into the kinesis assembly jar - Spark itself's core/connect/protobuf modules use protobuf 3, also shaded and relocated all protobuf 3 deps. Feel free to comment if you still have any concerns. [1] https://github.com/apache/spark/pull/41153 Thanks, Cheng Pan
