dwsmith1983 opened a new pull request, #60062: URL: https://github.com/apache/spark/pull/60062
### What changes were proposed in this pull request? Adds the `serialVersionUID` transition of `org.apache.spark.sql.types.Decimal` from Spark 4.3.0 (`4401554505066564464`) to this build to `UdfSerialization.auditedTransitions`, and a recorded 4.3.0 class descriptor to the `UdfSerializationSuite` test resources so the entry is backed by the released jar. The table's classes were audited against the published `spark-sql-api_2.13` jars of 4.0.5, 4.1.4, 4.2.1 and 4.3.0, loaded in an isolated class loader. Only `Decimal` in 4.3.0 has a new `serialVersionUID`. Like the releases already in the table, it declares no `serialVersionUID`, has no `readObject`/`writeObject`, is not `Externalizable`, and its fields (`_precision`, `_scale`, `longVal`, `decimalVal`) are identical. Every other audited class in these releases has a `serialVersionUID` that is either already a `streamSuid` in the table or equal to this build's. Following the existing resources, which only hold the classes whose descriptor first appeared in that release, the new `sql-types-classes-4.3.0.bin` holds just `Decimal`; 4.0.5, 4.1.4 and 4.2.1 add nothing new and get no resource. The same generator reproduces the committed `sql-types-classes-4.2.0.bin` byte for byte. #60056 (SPARK-60843) and #59159 (SPARK-59805) both change `Decimal`'s computed `serialVersionUID` on master. Whichever of these lands later needs to update the `localSuid` of both `Decimal` entries; `UdfSerializationSuite` fails until it does. ### Why are the changes needed? Spark 4.3.0 compiled `Decimal` with a different set of methods than 4.0 to 4.2 and master, so its computed `serialVersionUID` differs from both. Without a transition, a Scala UDF payload from a 4.3.0 client that carries a `Decimal` is rejected by a server built from master with `InvalidClassException: org.apache.spark.sql.types.Decimal; local class incompatible: stream classdesc serialVersionUID = 4401554505066564464, local class serialVersionUID = 6398599065815978361`. ### Does this PR introduce _any_ user-facing change? Yes, for Spark Connect: Scala UDF payloads from 4.3.0 clients that carry a `Decimal` are now accepted, as they already are from 4.0 to 4.2 clients. ### How was this patch tested? `UdfSerializationSuite`. With only the new 4.3.0 resource added, "audited transitions are backed by released producer descriptors" fails with the `InvalidClassException` above; with the new entry, the suite passes. ### Was this patch authored or co-authored using generative AI tooling? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
