perhaps this is https://issues.apache.org/jira/browse/SPARK-24578?
that was reported as a performance issue, not OOMs, but its in the exact
same part of the code and the change was to reduce the memory pressure
significantly.
On Mon, Jul 16, 2018 at 1:43 PM, Bryan Jeffrey
wrote:
> Hello.
>
> I
Hello.
I am working to move our system from Spark 2.1.0 to Spark 2.3.0. Our
system is running on Spark managed via Yarn. During the course of the move
I mirrored the settings to our new cluster. However, on the Spark 2.3.0
cluster with the same resource allocation I am seeing a number of
