[
https://issues.apache.org/jira/browse/SPARK-28697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-28697:
---------------------------------
Description:
{code}
spark-sql> create database func1;
Time taken: 0.095 seconds
spark-sql> use func1;
Time taken: 0.031 seconds
spark-sql> create table _(id int);
Time taken: 0.351 seconds
spark-sql> insert into _ values(1);
Time taken: 3.148 seconds
spark-sql> select * from _;
org.apache.hadoop.mapred.InvalidInputException: Input path does not exist:
hdfs://hacluster/user/sparkhive/warehouse/func1.db/_
at
org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:287)
{code}
But at HDFS side it is present
{code}
vm1:/opt/HA/C10/install/hadoop/nodemanager/bin # ./hdfs dfs -ls
/user/sparkhive/warehouse/func1.db
Found 2 items
drwxr-xr-x - root hadoop 0 2019-08-12 20:02
/user/sparkhive/warehouse/func1.db/_
{code}
was:
spark-sql> create database func1;
Time taken: 0.095 seconds
spark-sql> use func1;
Time taken: 0.031 seconds
spark-sql> create table _(id int);
Time taken: 0.351 seconds
spark-sql> insert into _ values(1);
Time taken: 3.148 seconds
spark-sql> select * from _;
org.apache.hadoop.mapred.InvalidInputException: Input path does not exist:
hdfs://hacluster/user/sparkhive/warehouse/func1.db/_
at
org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:287)
But at HDFS side it is present
vm1:/opt/HA/C10/install/hadoop/nodemanager/bin # ./hdfs dfs -ls
/user/sparkhive/warehouse/func1.db
Found 2 items
drwxr-xr-x - root hadoop 0 2019-08-12 20:02
/user/sparkhive/warehouse/func1.db/_
> select * from _; throws InvalidInputException and says path does not exists
> at HDFS side
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-28697
> URL: https://issues.apache.org/jira/browse/SPARK-28697
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: ABHISHEK KUMAR GUPTA
> Priority: Major
> Attachments: screenshot-1.png
>
>
> {code}
> spark-sql> create database func1;
> Time taken: 0.095 seconds
> spark-sql> use func1;
> Time taken: 0.031 seconds
> spark-sql> create table _(id int);
> Time taken: 0.351 seconds
> spark-sql> insert into _ values(1);
> Time taken: 3.148 seconds
> spark-sql> select * from _;
> org.apache.hadoop.mapred.InvalidInputException: Input path does not exist:
> hdfs://hacluster/user/sparkhive/warehouse/func1.db/_
> at
> org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:287)
> {code}
> But at HDFS side it is present
> {code}
> vm1:/opt/HA/C10/install/hadoop/nodemanager/bin # ./hdfs dfs -ls
> /user/sparkhive/warehouse/func1.db
> Found 2 items
> drwxr-xr-x - root hadoop 0 2019-08-12 20:02
> /user/sparkhive/warehouse/func1.db/_
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
