[
https://issues.apache.org/jira/browse/CARBONDATA-697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Babulal reassigned CARBONDATA-697:
----------------------------------
Assignee: Babulal
> single_pass is not used while doing data load
> ---------------------------------------------
>
> Key: CARBONDATA-697
> URL: https://issues.apache.org/jira/browse/CARBONDATA-697
> Project: CarbonData
> Issue Type: Bug
> Affects Versions: 0.1.1-incubating
> Environment: Spark 2.1
> Reporter: Ramakrishna
> Assignee: Babulal
> Fix For: 1.1.0-incubating
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> 1. 1st load with 'USE_KETTLE'='FALSE','SINGLE_PASS'='FALSE'
> 2. 2nd load with 'USE_KETTLE'='FALSE','SINGLE_PASS'='TRUE'
> Expected: GlobalDictionaryUtil should not be triggered in the 2nd time load
> Actual : GlobalDictionaryUtil was triggered
> Job Id (Job Group) ▾
> Description
> Submitted
> Duration
> Stages: Succeeded/Total Tasks (for all stages): Succeeded/Total
> 3 (891a750d-1e08-432f-bcf4-5501332d6d09)
> LOAD DATA inpath 'hdfs://hacluster/commbank/splits/FG2' into table
> flow_carbon
> options('USE_KETTLE'='FALSE','SINGLE_PASS'='TRUE','DELIMITER'=',',
> 'QUOTECHAR'='"','FILEHEADER'='aco_ac,ac_dte,txn_cnt,jrn_par,mfm_jrn_no,cbn_jrn_no,ibs_jrn_no,vch_no,vch_seq,srv_cde,cus_no,bus_cd_no,id_flg,cus_ac,bv_cde,bv_no,txn_dte,txn_time,txn_tlr,txn_bk,txn_br,ety_tlr,ety_bk,ety_br,bus_pss_no,chk_flg,chk_tlr,chk_jrn_no,bus_sys_no,bus_opr_cde,txn_sub_cde,fin_bus_cde,fin_bus_sub_cde,opt_prd_cde,chl,tml_id,sus_no,sus_seq,cho_seq,itm_itm,itm_sub,itm_sss,dc_flg,amt,bal,ccy,spv_flg,vch_vld_dte,pst_bk,pst_br,ec_flg,aco_tlr,opp_ac,opp_ac_nme,opp_bk,gen_flg,his_rec_sum_flg,his_flg,vch_typ,val_dte,opp_ac_flg,cmb_flg,ass_vch_flg,cus_pps_flg,bus_rmk_cde,vch_bus_rmk,tec_rmk_cde,vch_tec_rmk,rsv_ara,own_br,own_bk,gems_last_upd_d,gems_last_upd_d_bat,maps_date,maps_job,dt')
> collect at CarbonDataRDDFactory.scala:651 2017/02/06 15:23:05 41 s
> 1/1
> 3/3
> 2 (891a750d-1e08-432f-bcf4-5501332d6d09)
> LOAD DATA inpath 'hdfs://hacluster/commbank/splits/FG2' into table
> flow_carbon
> options('USE_KETTLE'='FALSE','SINGLE_PASS'='TRUE','DELIMITER'=',',
> 'QUOTECHAR'='"','FILEHEADER'='aco_ac,ac_dte,txn_cnt,jrn_par,mfm_jrn_no,cbn_jrn_no,ibs_jrn_no,vch_no,vch_seq,srv_cde,cus_no,bus_cd_no,id_flg,cus_ac,bv_cde,bv_no,txn_dte,txn_time,txn_tlr,txn_bk,txn_br,ety_tlr,ety_bk,ety_br,bus_pss_no,chk_flg,chk_tlr,chk_jrn_no,bus_sys_no,bus_opr_cde,txn_sub_cde,fin_bus_cde,fin_bus_sub_cde,opt_prd_cde,chl,tml_id,sus_no,sus_seq,cho_seq,itm_itm,itm_sub,itm_sss,dc_flg,amt,bal,ccy,spv_flg,vch_vld_dte,pst_bk,pst_br,ec_flg,aco_tlr,opp_ac,opp_ac_nme,opp_bk,gen_flg,his_rec_sum_flg,his_flg,vch_typ,val_dte,opp_ac_flg,cmb_flg,ass_vch_flg,cus_pps_flg,bus_rmk_cde,vch_bus_rmk,tec_rmk_cde,vch_tec_rmk,rsv_ara,own_br,own_bk,gems_last_upd_d,gems_last_upd_d_bat,maps_date,maps_job,dt')
> collect at GlobalDictionaryUtil.scala:742 2017/02/06 15:22:29 36 s
> 2/2
> 42/42
> 1 (c7706941-8184-40ab-9ceb-712af96e15e3)
> LOAD DATA inpath 'hdfs://hacluster/commbank/splits/FG2' into table
> flow_carbon
> options('USE_KETTLE'='FALSE','SINGLE_PASS'='FALSE','DELIMITER'=',',
> 'QUOTECHAR'='"','FILEHEADER'='aco_ac,ac_dte,txn_cnt,jrn_par,mfm_jrn_no,cbn_jrn_no,ibs_jrn_no,vch_no,vch_seq,srv_cde,cus_no,bus_cd_no,id_flg,cus_ac,bv_cde,bv_no,txn_dte,txn_time,txn_tlr,txn_bk,txn_br,ety_tlr,ety_bk,ety_br,bus_pss_no,chk_flg,chk_tlr,chk_jrn_no,bus_sys_no,bus_opr_cde,txn_sub_cde,fin_bus_cde,fin_bus_sub_cde,opt_prd_cde,chl,tml_id,sus_no,sus_seq,cho_seq,itm_itm,itm_sub,itm_sss,dc_flg,amt,bal,ccy,spv_flg,vch_vld_dte,pst_bk,pst_br,ec_flg,aco_tlr,opp_ac,opp_ac_nme,opp_bk,gen_flg,his_rec_sum_flg,his_flg,vch_typ,val_dte,opp_ac_flg,cmb_flg,ass_vch_flg,cus_pps_flg,bus_rmk_cde,vch_bus_rmk,tec_rmk_cde,vch_tec_rmk,rsv_ara,own_br,own_bk,gems_last_upd_d,gems_last_upd_d_bat,maps_date,maps_job,dt')
> collect at CarbonDataRDDFactory.scala:651 2017/02/06 15:21:42 47 s
> 1/1
> 3/3
> 0 (c7706941-8184-40ab-9ceb-712af96e15e3)
> LOAD DATA inpath 'hdfs://hacluster/commbank/splits/FG2' into table
> flow_carbon
> options('USE_KETTLE'='FALSE','SINGLE_PASS'='FALSE','DELIMITER'=',',
> 'QUOTECHAR'='"','FILEHEADER'='aco_ac,ac_dte,txn_cnt,jrn_par,mfm_jrn_no,cbn_jrn_no,ibs_jrn_no,vch_no,vch_seq,srv_cde,cus_no,bus_cd_no,id_flg,cus_ac,bv_cde,bv_no,txn_dte,txn_time,txn_tlr,txn_bk,txn_br,ety_tlr,ety_bk,ety_br,bus_pss_no,chk_flg,chk_tlr,chk_jrn_no,bus_sys_no,bus_opr_cde,txn_sub_cde,fin_bus_cde,fin_bus_sub_cde,opt_prd_cde,chl,tml_id,sus_no,sus_seq,cho_seq,itm_itm,itm_sub,itm_sss,dc_flg,amt,bal,ccy,spv_flg,vch_vld_dte,pst_bk,pst_br,ec_flg,aco_tlr,opp_ac,opp_ac_nme,opp_bk,gen_flg,his_rec_sum_flg,his_flg,vch_typ,val_dte,opp_ac_flg,cmb_flg,ass_vch_flg,cus_pps_flg,bus_rmk_cde,vch_bus_rmk,tec_rmk_cde,vch_tec_rmk,rsv_ara,own_br,own_bk,gems_last_upd_d,gems_last_upd_d_bat,maps_date,maps_job,dt')
> collect at GlobalDictionaryUtil.scala:742
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
