[
https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17693467#comment-17693467
]
kevinshin commented on SPARK-42550:
-----------------------------------
https://github.com/apache/kyuubi/issues/4416
> table directory will lost on hdfs when `INSERT OVERWRITE` faild
> ---------------------------------------------------------------
>
> Key: SPARK-42550
> URL: https://issues.apache.org/jira/browse/SPARK-42550
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.2.3
> Environment: spark 3.2.3 / HDP 3.1.4
> Reporter: kevinshin
> Priority: Critical
> Attachments: image-2023-02-24-15-21-55-273.png,
> image-2023-02-24-15-23-32-977.png, image-2023-02-24-15-25-57-770.png
>
>
> {color:#4c9aff}when a `{*}INSERT{*} OVERWRITE *TABLE`* statment faild during
> execution, the table's directory will be deleted. this is not happen in
> spark 3.2.1.{color}
> {color:#4c9aff}for example: {color}
> *CREATE* *TABLE* *IF* *NOT* *EXISTS* test.spark32_overwrite(amt1 {*}int{*})
> STORED *AS* ORC;
> *INSERT* OVERWRITE *TABLE* test.spark32_overwrite *select* 128;
> *CREATE* *TABLE* *IF* *NOT* *EXISTS* test.spark32_overwrite2(amt1 long)
> STORED *AS* ORC;
> *INSERT* OVERWRITE *TABLE* test.spark32_overwrite *select* 6000044164;
> *INSERT* OVERWRITE *TABLE* test.spark32_overwrite *select* amt1 *from*
> test.spark32_overwrite2; – {color:#de350b}*this will got Casting overflow
> exception*{color}
> {color:#4c9aff}and then :{color}
> *select* * *from* test.spark32_overwrite;
> {color:#4c9aff}will got error:{color}
> {color:#172b4d}java.io.FileNotFoundException{color}
> {color:#172b4d}!image-2023-02-24-15-21-55-273.png!{color}
> {color:#172b4d}the table's directory is losted. use `hdfs dfs -ls` cmd to
> check:{color}
> !image-2023-02-24-15-23-32-977.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
