JoshRosen commented on PR #37413: URL: https://github.com/apache/spark/pull/37413#issuecomment-1236173944
Hi @sos3k, We investigated your bug report and determined that the root-cause was a latent bug in `UnsafeHashedRelation` that was triggered more frequently following the backport of this PR. PR https://github.com/apache/spark/pull/35836 fixed a bug related to driver-side deserialization of `UnsafeHashedRelation`, but that fix was missing from DBR 9.1 and 10.4 (but is present in all newer versions). We have deployed a hotfix to backport https://github.com/apache/spark/pull/35836 into DBR 9.1 and 10.4. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
