[
https://issues.apache.org/jira/browse/SPARK-47961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugen Stoianovici updated SPARK-47961:
--------------------------------------
Component/s: Spark Core
> CREATE TABLE AS SELECT changes behaviour in SPARK 3.4.0
> -------------------------------------------------------
>
> Key: SPARK-47961
> URL: https://issues.apache.org/jira/browse/SPARK-47961
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, SQL
> Affects Versions: 3.4.0
> Reporter: Eugen Stoianovici
> Priority: Major
>
> SPARK-41859 changes the behaviour for `CREATE TABLE AS SELECT ...` from
> OVERWRITE to APPEND when {{spark.sql.legacy.allowNonEmptyLocationInCTAS}} is
> set to {{{}true{}}}:
> {{drop table if exists test_table;}}
> {{create table test_table location '/tmp/test_table' stored as parquet as
> select 1 as col union all select 2 as col;}}
> {{drop table if exists test_table;}}
> {{create table test_table location '/tmp/test_table' stored as parquet as
> select 3 as col union all select 4 as col;}}
> {{select * from test_table;}}
> This produces {3, 4} in Spark <3.4.0 and {1, 2, 3, 4} in Spark 3.4.0 and
> later. This is a silent change in
> {{spark.sql.legacy.allowNonEmptyLocationInCTAS}} behaviour which introduces
> wrong results in the user application
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
