Egor Pahomov created SPARK-18931: ------------------------------------ Summary: Create empty staging directory in partitioned table on insert Key: SPARK-18931 URL: https://issues.apache.org/jira/browse/SPARK-18931 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.0.2 Reporter: Egor Pahomov
CREATE TABLE temp.test_partitioning_4 ( num string ) PARTITIONED BY ( day string) stored as parquet On every INSERT INTO TABLE temp.test_partitioning_4 PARTITION (day) select day, count(*) as num from hss.session where year=2016 and month=4 group by day new directory ".hive-staging_hive_2016-12-19_15-55-11_298_3412488541559534475-4" created on HDFS. It's big issue, because I insert every day and bunch of empty dirs on HDFS is very bad for HDFS. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org