Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19497 @mridulm, I just checked thought the related changes and checked the tests pass on branch-2.1. Seems this PR will actually also allow the cases below: ```scala .saveAsNewAPIHadoopFile[...]("") .saveAsNewAPIHadoopFile[...]("::invalid:::") ``` Currently both are failed but seems this PR allows those cases: ``` Can not create a Path from an empty string java.lang.IllegalArgumentException: Can not create a Path from an empty string at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127) at org.apache.hadoop.fs.Path.<init>(Path.java:135) at org.apache.hadoop.fs.Path.<init>(Path.java:89) at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.absPathStagingDir(HadoopMapReduceCommitProtocol.scala:61) ... ``` ``` java.net.URISyntaxException: Relative path in absolute URI: ::invalid::: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ::invalid::: at org.apache.hadoop.fs.Path.initialize(Path.java:206) at org.apache.hadoop.fs.Path.<init>(Path.java:172) at org.apache.hadoop.fs.Path.<init>(Path.java:89) at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.absPathStagingDir(HadoopMapReduceCommitProtocol.scala:61) ... ``` I think we should protect these cases as this. For the cases for old one: ```scala .saveAsHadoopFile[...]("") .saveAsHadoopFile[...]("::invalid:::") ``` these looks failed fast (whether it was initially intended or not) and I guess this PR does not affect these: ``` Can not create a Path from an empty string java.lang.IllegalArgumentException: Can not create a Path from an empty string at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127) at org.apache.hadoop.fs.Path.<init>(Path.java:135) at org.apache.spark.internal.io.SparkHadoopWriterUtils$.createPathFromString(SparkHadoopWriterUtils.scala:54) ``` ``` java.net.URISyntaxException: Relative path in absolute URI: ::invalid::: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ::invalid::: at org.apache.hadoop.fs.Path.initialize(Path.java:206) at org.apache.hadoop.fs.Path.<init>(Path.java:172) at org.apache.spark.internal.io.SparkHadoopWriterUtils$.createPathFromString(SparkHadoopWriterUtils.scala:54) ```
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org