Hi,
The following query fails in 2.3.2:
scala> spark.range(10).write.saveAsTable("t1")
...
2018-09-29 20:48:06 ERROR FileOutputCommitter:314 - Mkdirs failed to create
file:/user/hive/warehouse/bucketed/_temporary/0
2018-09-29 20:48:07 ERROR Utils:91 - Aborting task
java.io.IOException: Mkdirs failed to create
file:/user/hive/warehouse/bucketed/_temporary/0/_temporary/attempt_20180929204807_0000_m_000003_0
(exists=false, cwd=file:/Users/jacek/dev/apps/spark-2.3.2-bin-hadoop2.7)
at
org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:455)
at
org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:440)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911)
While it works fine in 2.3.1.
Could anybody explain the change in behaviour in 2.3.2? The commit / the
JIRA issue would be even nicer. Thanks.
Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Mastering Spark SQL https://bit.ly/mastering-spark-sql
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
Follow me at https://twitter.com/jaceklaskowski
