Xiangrui Meng created SPARK-27588: ------------------------------------- Summary: Fail fast if binary file data source will load a file that is bigger than 2GB Key: SPARK-27588 URL: https://issues.apache.org/jira/browse/SPARK-27588 Project: Spark Issue Type: Story Components: SQL Affects Versions: 3.0.0 Reporter: Xiangrui Meng
We use BinaryType to store file content, if a file is bigger than 2GB, we should fail the job without trying to load the file that increases the error waiting time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org