[
https://issues.apache.org/jira/browse/SPARK-28697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905144#comment-16905144
]
Ajith S edited comment on SPARK-28697 at 8/12/19 12:44 PM:
-----------------------------------------------------------
!screenshot-1.png!
Here due to
org.apache.hadoop.fs.FileSystem#globStatus(org.apache.hadoop.fs.Path,
org.apache.hadoop.fs.PathFilter), it return null matches when folder name
starts with _
was (Author: ajithshetty):
!screenshot-1.png!
Here due to
org.apache.hadoop.fs.FileSystem#globStatus(org.apache.hadoop.fs.Path,
org.apache.hadoop.fs.PathFilter), it return 0 matches when folder name starts
with _
> select * from _; throws InvalidInputException and says path does not exists
> at HDFS side
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-28697
> URL: https://issues.apache.org/jira/browse/SPARK-28697
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: ABHISHEK KUMAR GUPTA
> Priority: Major
> Attachments: screenshot-1.png
>
>
> spark-sql> create database func1;
> Time taken: 0.095 seconds
> spark-sql> use func1;
> Time taken: 0.031 seconds
> spark-sql> create table _(id int);
> Time taken: 0.351 seconds
> spark-sql> insert into _ values(1);
> Time taken: 3.148 seconds
> spark-sql> select * from _;
> org.apache.hadoop.mapred.InvalidInputException: Input path does not exist:
> hdfs://hacluster/user/sparkhive/warehouse/func1.db/_
> at
> org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:287)
> But at HDFS side it is present
> vm1:/opt/HA/C10/install/hadoop/nodemanager/bin # ./hdfs dfs -ls
> /user/sparkhive/warehouse/func1.db
> Found 2 items
> drwxr-xr-x - root hadoop 0 2019-08-12 20:02
> /user/sparkhive/warehouse/func1.db/_
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
