Vincent Tran created IMPALA-8734: ------------------------------------ Summary: ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata Key: IMPALA-8734 URL: https://issues.apache.org/jira/browse/IMPALA-8734 Project: IMPALA Issue Type: Improvement Reporter: Vincent Tran
More particularly, if one needs to change the data representation of NULL in the tblproperties, an IM is needed for table to be readable with the new `serialization.null.format` To setup, create a table and dfs -put some data files into hdfs with some null strings: {noformat} [localhost:21000] default> create table v.t1(c1 string); Query: create table v.t1(c1 string) +-------------------------+ | summary | +-------------------------+ | Table has been created. | +-------------------------+ Fetched 1 row(s) in 0.51s [localhost:21000] default> show create table v.t1; Query: show create table v.t1 +----------------------------------------------------------+ | result | +----------------------------------------------------------+ | CREATE TABLE v.t1 ( | | c1 STRING | | ) | | STORED AS TEXTFILE | | LOCATION 'hdfs://localhost:20500/test-warehouse/v.db/t1' | | | +----------------------------------------------------------+ Fetched 1 row(s) in 4.38s [localhost:21000] default> !printf '\nnull\n' > f; -------- Executed in 0.00s [localhost:21000] default> !hdfs dfs -put f /test-warehouse/v.db/t1/f1; -------- Executed in 1.98s{noformat} Repro: {noformat} [localhost:21000] default> select * from v.t1; Query: select * from v.t1 Query submitted at: 2019-06-27 12:48:13 (Coordinator: http://blackbox.vpc.cloudera.com:25000) Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=8b4532852b10613e:e9e4847500000000 +------+ | c1 | +------+ | | | null | +------+ Fetched 2 row(s) in 4.77s [localhost:21000] default> alter table v.t1 set tblproperties("serialization.null.format"="null"); Query: alter table v.t1 set tblproperties("serialization.null.format"="null") +----------------+ | summary | +----------------+ | Updated table. | +----------------+ Fetched 1 row(s) in 0.05s [localhost:21000] default> select * from v.t1; Query: select * from v.t1 Query submitted at: 2019-06-27 12:48:39 (Coordinator: http://blackbox.vpc.cloudera.com:25000) Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=e54749def520fa97:9800728300000000 +------+ | c1 | +------+ | | | null | +------+ Fetched 2 row(s) in 0.11s [localhost:21000] default> invalidate metadata v.t1; Query: invalidate metadata v.t1 Query submitted at: 2019-06-27 12:48:50 (Coordinator: http://blackbox.vpc.cloudera.com:25000) Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=9d41e48bc8d5bad3:8020514f00000000 Fetched 0 row(s) in 0.01s [localhost:21000] default> select * from v.t1; Query: select * from v.t1 Query submitted at: 2019-06-27 12:48:51 (Coordinator: http://blackbox.vpc.cloudera.com:25000) Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=0446c2c65af3400a:3e3a484900000000 +------+ | c1 | +------+ | | | NULL | +------+ Fetched 2 row(s) in 4.48s [localhost:21000] default>{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)