Bao Yunz created SPARK-26407:
--------------------------------
Summary: Select result is incorrect when add a directory named
with k=v to the table path of external non-partition table
Key: SPARK-26407
URL: https://issues.apache.org/jira/browse/SPARK-26407
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.4.0, 2.3.2
Reporter: Bao Yunz
Scene 1
Create a external non-partition table, in which location path has a directory
named with "part=1", for example. Then desc the table, we will find the string
"part" is showed in table column. when insert the table with data which has
same column with target table , will throw a exception that target table has
different column number with the inserted data.
Scene 2
Create a external non-partition table, which location path is empty. After
several times insert operation, we add a directory named with "part=1" in the
table location directory. Then do insert and select operation, we will find the
scan path is changed to "tablePath/part=1"，so that we will get a wrong result.
It seems that the existing logic of spark will process this kind of table like
a partition table. But when we do show partitions operation, it will throw the
exception that the table is not partitioned, which is confusing。We believe that
the normal logic should be that if a table is a non-partitioned table, the
folder under tablePath should not change its basic properties.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
