Question of List Bucketing Table

[guoqing0...@yahoo.com.hk]

Hi all , 
I try to create List Bucketing Table in Hive 0.13.1  as bellow , 

set hive.mapred.supports.subdirectories=true;
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.input.format=org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
set hive.merge.smallfiles.avgsize=200;
set mapred.input.dir.recursive=true;
set hive.merge.mapfiles=false;
set hive.merge.mapredfiles=false;
drop table t_gg_tmp_skew;
create table t_gg_tmp_skew (
stockout_guid   string,
stockout_code   string,
comp_guid   string,
shop_guid   string,
sour_code_guid  string
)
PARTITIONED BY(stockout_type string);
alter table t_gg_tmp_skew skewed by (shop_guid) on 
('01F03452-1CCB-4FCA-909C-0B4D6DE1297F') stored as DIRECTORIES;

insert overwrite table t_gg_tmp_skew partition (stockout_type='SR')
select
stockout_guid   ,
stockout_code   ,
comp_guid   ,
shop_guid   ,
sour_code_guid
from t_gg_tmp where stockout_type='SR' ;

desc formatted t_gg_tmp_skew partition (stockout_type='SR')

The directory struct as bellow (only 1 item) :
/user/hive/warehouse/t_gg_tmp_skew/stockout_type=SR/HIVE_DEFAULT_LIST_BUCKETING_DIR_NAME
But i think it should had a sub-directory like shop_guid=xxx
/user/hive/warehouse/t_gg_tmp_skew/stockout_type=SR/shop_guid=xxx

I guess lost Skewed Value to Truncated Path , please help to point out the 
mistake , thank you very much.



guoqing0...@yahoo.com.hk

Hive on Spark VS Spark SQL

[guoqing0...@yahoo.com.hk]

Hive on Spark and SparkSQL which should be better , and what are the key 
characteristics and the advantages and the disadvantages between ?



guoqing0...@yahoo.com.hk