Thats not the expected behavior. What version of Spark?
On Mon, Aug 24, 2015 at 1:32 AM, Kevin Jung itsjb.j...@samsung.com wrote:
When I store DataFrame as table with command saveAsTable and then
execute DROP TABLE in SparkSQL, it doesn't actually delete files in hive
warehouse.
The table
When I store DataFrame as table with command saveAsTable and then execute
DROP TABLE in SparkSQL, it doesn't actually delete files in hive warehouse.
The table disappears from a table list but the data files are still alive.
Because of this, I can't saveAsTable with a same name before dropping
a table with same location and same name. If I update
DBS table in metastoredb to set default database URI to HDFS1, it works
perfectly.
Kevin
--- Original Message ---
Sender : Michael Armbrustmich...@databricks.com
Date : 2015-08-25 00:43 (GMT+09:00)
Title : Re: Drop table and Hive
