subject:"\[GitHub\] \[spark\] JoshRosen commented on pull request #37413\: \[SPARK\-39983\]\[CORE\]\[SQL\] Do not cache unserialized broadcast relations on the driver"

[GitHub] [spark] JoshRosen commented on pull request #37413: [SPARK-39983][CORE][SQL] Do not cache unserialized broadcast relations on the driver

2022-09-03 Thread GitBox

JoshRosen commented on PR #37413: URL: https://github.com/apache/spark/pull/37413#issuecomment-1236173944 Hi @sos3k, We investigated your bug report and determined that the root-cause was a latent bug in `UnsafeHashedRelation` that was triggered more frequently following the

[GitHub] [spark] JoshRosen commented on pull request #37413: [SPARK-39983][CORE][SQL] Do not cache unserialized broadcast relations on the driver

2022-08-09 Thread GitBox

JoshRosen commented on PR #37413: URL: https://github.com/apache/spark/pull/37413#issuecomment-1209009482 > QQ @JoshRosen, @alex-balikov - if the expectation is that the variable can be recreated (if missing) at driver - with that being a remote possibility, do we want to make it a