Shay Elbaz created SPARK-26438: ---------------------------------- Summary: Driver waits to spark.sql.broadcastTimeout before throwing OutOfMemoryError - is this by design? Key: SPARK-26438 URL: https://issues.apache.org/jira/browse/SPARK-26438 Project: Spark Issue Type: Question Components: Spark Core, SQL Affects Versions: 2.3.0 Reporter: Shay Elbaz
When broadcasting too large DataFrame, the driver does not fail immediately when the broadcast thread throws OutOfMemoryError. Instead it waits for `spark.sql.broadcastTimeout` to meet. Is that by design or a bug? -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org