Hi, The behaviour is the same for me in Scala and Python, so posting here in Python. When I use DataFrame.saveAsTable with the path option, I expect an external Hive table to be created at the specified path. Specifically, when I call:
>>> df.saveAsTable(..., path="/tmp/test") I expect an external Hive table to be created pointing to /tmp/test which would contain the data in df. However, running locally on my Mac, I get an error indicating that Spark tried to create a managed table in the location of the Hive warehouse: ERROR RetryingHMSHandler: MetaException(message:file:/user/hive/warehouse/savetable is not a directory or unable to create one) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1239) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1294) Am I wrong to expect that Spark create an external table in this case? What is the expected behaviour of saveAsTable with the path option? Setup: running spark locally with spark 1.3.0. Kind Regards, Tom
