SHREELEKHYA GAMPA created CARBONDATA-4282:
---------------------------------------------
Summary: Issues with table having complex columns related to long
string, SI, local dictionary
Key: CARBONDATA-4282
URL: https://issues.apache.org/jira/browse/CARBONDATA-4282
Project: CarbonData
Issue Type: Bug
Reporter: SHREELEKHYA GAMPA
*1. Insert/load fails after alter add complex column if table contains long
string columns.*
[Steps] :-
DROP TABLE IF EXISTS alter_com;
CREATE TABLE alter_com(intfield int,EDUCATED string ,rankk string ) STORED AS
carbondata
TBLPROPERTIES('inverted_index'='intField','sort_columns'='intField','TABLE_BLOCKSIZE'=
'256
MB','TABLE_BLOCKLET_SIZE'='8','SORT_SCOPE'='no_sort','COLUMN_META_CACHE'='rankk','carbon.column.compressor'='gzip','long_string_columns'='rankk','table_page_size_inmb'='1');
insert into alter_com values(1,'cse','xi'); select * from alter_com limit 1;
ALTER TABLE alter_com ADD COLUMNS(map1 Map<short,int>, map2 Map<long,double>,
map3 Map<decimal(3,2),string>, map4 Map<char(5),varchar(50)>, map5
Map<boolean,date>,map6 Map<string,timestamp>,map7 map<string,array<string>>,
map8 map<string,struct<d:int, s:struct<im:string>>>);
ALTER TABLE alter_com SET TBLPROPERTIES('long_string_columns'='EDUCATED');
insert into alter_com values(1,'ece','x', map(1,2),map(3,2.34),
map(1.23,'hello'),map('abc','def'),
map(true,'2017-02-01'),map('time','2018-02-01
02:00:00.0'),map('ph',array(1,2)),
map('a',named_struct('d',23,'s',named_struct('im','sh'))));
[Expected Result] :- insert/load should be success after alter add map column
,if table contains long string columns
*2. create index on array of complex column (map/struct) throws null pointer
exception instead of correct error message.*
[Steps] :-
drop table if exists strarmap1; create table strarmap1(id int,name string,str
struct<intval:int,tm:timestamp,map1:map<int,string>>,arr
array<map<string,timestamp>>) stored as carbondata
tblproperties('inverted_index'='name','sort_columns'='name','TABLE_BLOCKSIZE'=
'256 MB','TABLE_BLOCKLET_SIZE'='8','CACHE_LEVEL'='BLOCKLET'); load data inpath
'hdfs://hacluster/chetan/strarmap1.csv' into table strarmap1
options('fileheader'='id,name,str,arr','COMPLEX_DELIMITER_LEVEL_3'='#','COMPLEX_DELIMITER_LEVEL_2'='$','COMPLEX_DELIMITER_LEVEL_1'='&','BAD_RECORDS_ACTION'='FORCE');
CREATE INDEX index2 ON TABLE strarmap1 (arr) as 'carbondata'
properties('sort_scope'='global_sort','global_sort_partitions'='3');
[Expected Result] :- create index on array of map(string,timestamp) should
thrown correct validation error message.
[Actual Issue]:- create index on array of map(string,timestamp) throws null
pointer exception instead of correct error message
*3. alter table property local dictionary inlcude/exclude with newly added map
column is failing.*
[Steps] :-
drop table if exists strarmap1; create table strarmap1(id int,name string,str
struct<intval:int,tm:timestamp,map1:map<int,string>>,arr
array<map<string,timestamp>>) stored as carbondata
tblproperties('inverted_index'='name','sort_columns'='name','local_dictionary_enable'='false','local_dictionary_include'='map1','local_dictionary_exclude'='str,arr','local_dictionary_threshold'='1000');
ALTER TABLE strarmap1 ADD COLUMNS(map1 Map<short,int>, map2 Map<long,double>,
map3 Map<decimal(3,2),string>, map4 Map<char(5),varchar(50)>, map5
Map<boolean,date>,map6 Map<string,timestamp>,map7 map<string,array<string>>,
map8 map<string,struct<d:int, s:struct<im:string>>>); ALTER TABLE strarmap1 SET
TBLPROPERTIES('LOCAL_DICTIONARY_ENABLE'='true','local_dictionary_include'='map4','local_dictionary_threshold'='1000');
[Expected Result] :- alter table property local dictionary inlcude/exclude with
newly added map column should be success
[Actual Issue]:- alter table property local dictionary inlcude/exclude with
newly added map column is failing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
