[ https://issues.apache.org/jira/browse/SPARK-28841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16912719#comment-16912719 ]
Shixiong Zhu edited comment on SPARK-28841 at 8/21/19 10:00 PM: ---------------------------------------------------------------- Okey, at least the following codes are legit but failing. {code} scala> :paste // Entering paste mode (ctrl-D to finish) import org.apache.hadoop.fs._ val fs = new Path("/").getFileSystem(spark.sessionState.newHadoopConf()) fs.create(new Path("/tmp/test/foo:bar")).close() // Exiting paste mode, now interpreting. java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: .foo:bar.crc at org.apache.hadoop.fs.Path.initialize(Path.java:205) at org.apache.hadoop.fs.Path.<init>(Path.java:171) at org.apache.hadoop.fs.Path.<init>(Path.java:93) at org.apache.hadoop.fs.ChecksumFileSystem.getChecksumFile(ChecksumFileSystem.java:90) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:402) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:461) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:440) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:892) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:789) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:778) ... 59 elided Caused by: java.net.URISyntaxException: Relative path in absolute URI: .foo:bar.crc at java.net.URI.checkPath(URI.java:1823) at java.net.URI.<init>(URI.java:745) at org.apache.hadoop.fs.Path.initialize(Path.java:202) ... 69 more {code} was (Author: zsxwing): Okey, at least the following codes are legit but failing. {code} scala> :paste // Entering paste mode (ctrl-D to finish) import org.apache.hadoop.fs._ val fs = new Path("/").getFileSystem(spark.sessionState.newHadoopConf()) fs.create(new Path("/tmp/test/foo:bar")).close() spark.read.text("/tmp/test/foo?bar") // Exiting paste mode, now interpreting. java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: .foo:bar.crc at org.apache.hadoop.fs.Path.initialize(Path.java:205) at org.apache.hadoop.fs.Path.<init>(Path.java:171) at org.apache.hadoop.fs.Path.<init>(Path.java:93) at org.apache.hadoop.fs.ChecksumFileSystem.getChecksumFile(ChecksumFileSystem.java:90) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:402) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:461) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:440) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:892) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:789) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:778) ... 59 elided Caused by: java.net.URISyntaxException: Relative path in absolute URI: .foo:bar.crc at java.net.URI.checkPath(URI.java:1823) at java.net.URI.<init>(URI.java:745) at org.apache.hadoop.fs.Path.initialize(Path.java:202) ... 69 more {code} > Spark cannot read a relative path containing ":" > ------------------------------------------------ > > Key: SPARK-28841 > URL: https://issues.apache.org/jira/browse/SPARK-28841 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.3 > Reporter: Shixiong Zhu > Priority: Major > > Reproducer: > {code} > spark.read.parquet("test:test") > {code} > Error: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: test:test > {code} > This is actually a Hadoop issue since the error is thrown from "new > Path("test:test")". I'm creating this ticket to see if we can work around > this issue in Spark. -- This message was sent by Atlassian Jira (v8.3.2#803003) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org