[
https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429068#comment-15429068
]
Peter Vary commented on HIVE-14146:
-----------------------------------
It might be confusing, but here is what the patch changes:
describe escape_comments_tbl1;
{noformat:title=Beeline original}
+--------------------------+-----------------------+-----------------------+--+
| col_name | data_type | comment |
+--------------------------+-----------------------+-----------------------+--+
| col1 | string | a |
| b | NULL | NULL |
| p1 | string | a |
| b | NULL | NULL |
| | NULL | NULL |
| # Partition Information | NULL | NULL |
| # col_name | data_type | comment |
| | NULL | NULL |
| p1 | string | a |
| b | NULL | NULL |
+--------------------------+-----------------------+-----------------------+--+
{noformat}
{noformat:title=Beeline patch}
+--------------------------+-----------------------+-----------------------+--+
| col_name | data_type | comment |
+--------------------------+-----------------------+-----------------------+--+
| col1 | string | a\nb |
| p1 | string | a\nb |
| | NULL | NULL |
| # Partition Information | NULL | NULL |
| # col_name | data_type | comment |
| | NULL | NULL |
| p1 | string | a\nb |
+--------------------------+-----------------------+-----------------------+--+
{noformat}
{noformat:title=Cli original}
col1 string a
b
p1 string a
b
# Partition Information
# col_name data_type comment
p1 string a
b
{noformat}
{noformat:title=Cli patch}
col1 string a
b
p1 string a
b
# Partition Information
# col_name data_type comment
p1 string a
b
{noformat}
describe formatted escape_comments_tbl1;
{noformat:title=Beeline original}
+-------------------------------+----------------------------------------------------+-----------------------+--+
| col_name | data_type
| comment |
+-------------------------------+----------------------------------------------------+-----------------------+--+
| # col_name | data_type
| comment |
| | NULL
| NULL |
| col1 | string
| a |
| b | NULL
| NULL |
| | NULL
| NULL |
| # Partition Information | NULL
| NULL |
| # col_name | data_type
| comment |
| | NULL
| NULL |
| p1 | string
| a |
| b | NULL
| NULL |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | default
| NULL |
| Owner: | petervary
| NULL |
| CreateTime: | Fri Jul 01 06:40:51 PDT 2016
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Retention: | 0
| NULL |
| Location: |
file:/Users/petervary/data/apache/hive/warehouse/escape_comments_tbl1 | NULL
|
| Table Type: | MANAGED_TABLE
| NULL |
| Table Parameters: | NULL
| NULL |
| | comment
| a\nb |
| | transient_lastDdlTime
| 1467380451 |
| | NULL
| NULL |
| # Storage Information | NULL
| NULL |
| SerDe Library: |
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL |
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat
| NULL |
| OutputFormat: |
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
|
| Compressed: | No
| NULL |
| Num Buckets: | -1
| NULL |
| Bucket Columns: | []
| NULL |
| Sort Columns: | []
| NULL |
| Storage Desc Params: | NULL
| NULL |
| | serialization.format
| 1 |
+-------------------------------+----------------------------------------------------+-----------------------+--+
{noformat}
{noformat:title=Beeline patch}
+-------------------------------+----------------------------------------------------+-----------------------+--+
| col_name | data_type
| comment |
+-------------------------------+----------------------------------------------------+-----------------------+--+
| # col_name | data_type
| comment |
| | NULL
| NULL |
| col1 | string
| a\nb |
| | NULL
| NULL |
| # Partition Information | NULL
| NULL |
| # col_name | data_type
| comment |
| | NULL
| NULL |
| p1 | string
| a\nb |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | default
| NULL |
| Owner: | petervary
| NULL |
| CreateTime: | Fri Jul 01 06:40:51 PDT 2016
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Retention: | 0
| NULL |
| Location: |
file:/Users/petervary/data/apache/hive/warehouse/escape_comments_tbl1 | NULL
|
| Table Type: | MANAGED_TABLE
| NULL |
| Table Parameters: | NULL
| NULL |
| | comment
| a\nb |
| | transient_lastDdlTime
| 1467380451 |
| | NULL
| NULL |
| # Storage Information | NULL
| NULL |
| SerDe Library: |
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL |
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat
| NULL |
| OutputFormat: |
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
|
| Compressed: | No
| NULL |
| Num Buckets: | -1
| NULL |
| Bucket Columns: | []
| NULL |
| Sort Columns: | []
| NULL |
| Storage Desc Params: | NULL
| NULL |
| | serialization.format
| 1 |
+-------------------------------+----------------------------------------------------+-----------------------+--+
{noformat}
{noformat:title=Cli original}
# col_name data_type comment
col1 string a
b
# col_name data_type comment
p1 string a
b
# Detailed Table Information
Database: default
Owner: petervary
CreateTime: Fri Jul 01 06:40:51 PDT 2016
LastAccessTime: UNKNOWN
Retention: 0
Location:
file:/Users/petervary/data/apache/hive/warehouse/escape_comments_tbl1
Table Type: MANAGED_TABLE
Table Parameters:
comment a\nb
transient_lastDdlTime 1467380451
# Storage Information
SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
InputFormat: org.apache.hadoop.mapred.TextInputFormat
OutputFormat:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Compressed: No
Num Buckets: -1
Bucket Columns: []
Sort Columns: []
Storage Desc Params:
serialization.format 1
{noformat}
{noformat:title=Cli patch}
# col_name data_type comment
col1 string a
b
# Partition Information
# col_name data_type comment
p1 string a
b
# Detailed Table Information
Database: default
Owner: petervary
CreateTime: Fri Jul 01 06:40:51 PDT 2016
LastAccessTime: UNKNOWN
Retention: 0
Location:
file:/Users/petervary/data/apache/hive/warehouse/escape_comments_tbl1
Table Type: MANAGED_TABLE
Table Parameters:
comment a
b
transient_lastDdlTime 1467380451
# Storage Information
SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
InputFormat: org.apache.hadoop.mapred.TextInputFormat
OutputFormat:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Compressed: No
Num Buckets: -1
Bucket Columns: []
Sort Columns: []
Storage Desc Params:
serialization.format 1
{noformat}
describe pretty escape_comments_tbl1;
{noformat:title=Beeline original}
+--------------------------+-----------------+----------+--+
| col_name | data_type | comment |
+--------------------------+-----------------+----------+--+
| col_name | data_type | comment |
| | | |
| col1 | string | a |
| | | b |
| p1 | string | a |
| | | b |
| | NULL | NULL |
| # Partition Information | NULL | NULL |
| col_name | data_type | comment |
| | | |
| p1 | string | a |
| | | b |
+--------------------------+-----------------+----------+--+
{noformat}
{noformat:title=Beeline patch}
+--------------------------+-----------------+----------+--+
| col_name | data_type | comment |
+--------------------------+-----------------+----------+--+
| col_name | data_type | comment |
| | | |
| col1 | string | a |
| | | b |
| p1 | string | a |
| | | b |
| | NULL | NULL |
| # Partition Information | NULL | NULL |
| col_name | data_type | comment |
| | | |
| p1 | string | a |
| | | b |
+--------------------------+-----------------+----------+--+
{noformat}
{noformat:title=Cli original}
col_name data_type comment
col1 string a
b
p1 string a
b
# Partition Information
col_name data_type comment
p1 string a
b
{noformat}
{noformat:title=Cli patch}
col_name data_type comment
col1 string a
b
p1 string a
b
# Partition Information
col_name data_type comment
p1 string a
b
{noformat}
describe database escape_comments_db;
{noformat:title=Beeline original}
+---------------------+----------------------------------------------------+------------+-------------+-------------+-------------+--+
2 rows selected (0.054 seconds)
| db_name | comment |
location | owner_name | owner_type | parameters |
+---------------------+----------------------------------------------------+------------+-------------+-------------+-------------+--+
| escape_comments_db | a |
NULL | NULL | NULL | NULL |
| b |
file:/Users/petervary/data/apache/hive/warehouse/escape_comments_db.db |
petervary | USER | | NULL |
+---------------------+----------------------------------------------------+------------+-------------+-------------+-------------+--+
{noformat}
{noformat:title=Beeline patch}
+---------------------+----------+----------------------------------------------------+-------------+-------------+-------------+--+
| db_name | comment | location
| owner_name | owner_type | parameters |
+---------------------+----------+----------------------------------------------------+-------------+-------------+-------------+--+
| escape_comments_db | a\nb |
file:/Users/petervary/data/apache/hive/warehouse/escape_comments_db.db |
petervary | USER | |
+---------------------+----------+----------------------------------------------------+-------------+-------------+-------------+--+
{noformat}
{noformat:title=Cli original}
escape_comments_db a
b file:/Users/petervary/data/apache/hive/warehouse/escape_comments_db.db
petervary USER
{noformat}
{noformat:title=Cli patch}
escape_comments_db a\nb
file:/Users/petervary/data/apache/hive/warehouse/escape_comments_db.db
petervary USER
{noformat}
describe database extended escape_comments_db;
{noformat:title=Beeline original}
escape_comments_db a
b file:/Users/petervary/data/apache/hive/warehouse/escape_comments_db.db
petervary USER
{noformat}
{noformat:title=Beeline patch}
+---------------------+----------+----------------------------------------------------+-------------+-------------+-------------+--+
| db_name | comment | location
| owner_name | owner_type | parameters |
+---------------------+----------+----------------------------------------------------+-------------+-------------+-------------+--+
| escape_comments_db | a\nb |
file:/Users/petervary/data/apache/hive/warehouse/escape_comments_db.db |
petervary | USER | |
+---------------------+----------+----------------------------------------------------+-------------+-------------+-------------+--+
{noformat}
{noformat:title=Cli original}
escape_comments_db a
b file:/Users/petervary/data/apache/hive/warehouse/escape_comments_db.db
petervary USER
{noformat}
{noformat:title=Cli patch}
escape_comments_db a\nb
file:/Users/petervary/data/apache/hive/warehouse/escape_comments_db.db
petervary USER
{noformat}
describe formatted escape_comments_view1;
{noformat:title=Beeline original}
+-------------------------------+----------------------------------------------------+-----------------------+--+
| col_name | data_type
| comment |
+-------------------------------+----------------------------------------------------+-----------------------+--+
| # col_name | data_type
| comment |
| | NULL
| NULL |
| col1 | string
| a |
| b | NULL
| NULL |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | default
| NULL |
| Owner: | petervary
| NULL |
| CreateTime: | Fri Jul 01 06:40:51 PDT 2016
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Retention: | 0
| NULL |
| Table Type: | VIRTUAL_VIEW
| NULL |
| Table Parameters: | NULL
| NULL |
| | comment
| a\nb |
| | transient_lastDdlTime
| 1467380451 |
| | NULL
| NULL |
| # Storage Information | NULL
| NULL |
| SerDe Library: | null
| NULL |
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat
| NULL |
| OutputFormat: |
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
|
| Compressed: | No
| NULL |
| Num Buckets: | -1
| NULL |
| Bucket Columns: | []
| NULL |
| Sort Columns: | []
| NULL |
| | NULL
| NULL |
| # View Information | NULL
| NULL |
| View Original Text: | select col1 from escape_comments_tbl1
| NULL |
| View Expanded Text: | SELECT `col1` AS `col1` FROM (select
`escape_comments_tbl1`.`col1` from `default`.`escape_comments_tbl1`)
`default.escape_comments_view1` | NULL |
+-------------------------------+----------------------------------------------------+-----------------------+--+
{noformat}
{noformat:title=Beeline patch}
+-------------------------------+----------------------------------------------------+-----------------------+--+
| col_name | data_type
| comment |
+-------------------------------+----------------------------------------------------+-----------------------+--+
| # col_name | data_type
| comment |
| | NULL
| NULL |
| col1 | string
| a\nb |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | default
| NULL |
| Owner: | petervary
| NULL |
| CreateTime: | Fri Jul 01 06:40:51 PDT 2016
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Retention: | 0
| NULL |
| Table Type: | VIRTUAL_VIEW
| NULL |
| Table Parameters: | NULL
| NULL |
| | comment
| a\nb |
| | transient_lastDdlTime
| 1467380451 |
| | NULL
| NULL |
| # Storage Information | NULL
| NULL |
| SerDe Library: | null
| NULL |
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat
| NULL |
| OutputFormat: |
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
|
| Compressed: | No
| NULL |
| Num Buckets: | -1
| NULL |
| Bucket Columns: | []
| NULL |
| Sort Columns: | []
| NULL |
| | NULL
| NULL |
| # View Information | NULL
| NULL |
| View Original Text: | select col1 from escape_comments_tbl1
| NULL |
| View Expanded Text: | SELECT `col1` AS `col1` FROM (select
`escape_comments_tbl1`.`col1` from `default`.`escape_comments_tbl1`)
`default.escape_comments_view1` | NULL |
+-------------------------------+----------------------------------------------------+-----------------------+--+
{noformat}
{noformat:title=Cli original}
# col_name data_type comment
col1 string a
b
# Detailed Table Information
Database: default
Owner: petervary
CreateTime: Fri Jul 01 06:40:51 PDT 2016
LastAccessTime: UNKNOWN
Retention: 0
Table Type: VIRTUAL_VIEW
Table Parameters:
comment a\nb
transient_lastDdlTime 1467380451
# Storage Information
SerDe Library: null
InputFormat: org.apache.hadoop.mapred.TextInputFormat
OutputFormat:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Compressed: No
Num Buckets: -1
Bucket Columns: []
Sort Columns: []
# View Information
View Original Text: select col1 from escape_comments_tbl1
View Expanded Text: SELECT `col1` AS `col1` FROM (select
`escape_comments_tbl1`.`col1` from `default`.`escape_comments_tbl1`)
`default.escape_comments_view1`
{noformat}
{noformat:title=Cli patch}
# col_name data_type comment
col1 string a
b
# Detailed Table Information
Database: default
Owner: petervary
CreateTime: Fri Jul 01 06:40:51 PDT 2016
LastAccessTime: UNKNOWN
Retention: 0
Table Type: VIRTUAL_VIEW
Table Parameters:
comment a
b
transient_lastDdlTime 1467380451
# Storage Information
SerDe Library: null
InputFormat: org.apache.hadoop.mapred.TextInputFormat
OutputFormat:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Compressed: No
Num Buckets: -1
Bucket Columns: []
Sort Columns: []
# View Information
View Original Text: select col1 from escape_comments_tbl1
View Expanded Text: SELECT `col1` AS `col1` FROM (select
`escape_comments_tbl1`.`col1` from `default`.`escape_comments_tbl1`)
`default.escape_comments_view1`
{noformat}
show create table escape_comments_tbl1;
{noformat:title=Beeline original}
+----------------------------------------------------+--+
| createtab_stmt |
+----------------------------------------------------+--+
| CREATE TABLE `escape_comments_tbl1`( |
| `col1` string COMMENT 'a |
| b') |
| COMMENT 'a |
| b' |
| PARTITIONED BY ( |
| `p1` string COMMENT 'a |
| b') |
| ROW FORMAT SERDE |
| 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' |
| STORED AS INPUTFORMAT |
| 'org.apache.hadoop.mapred.TextInputFormat' |
| OUTPUTFORMAT |
| 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' |
| LOCATION |
| 'file:/Users/petervary/data/apache/hive/warehouse/escape_comments_tbl1' |
| TBLPROPERTIES ( |
| 'transient_lastDdlTime'='1467380451') |
+----------------------------------------------------+--+
{noformat}
{noformat:title=Beeline patch}
+----------------------------------------------------+--+
| createtab_stmt |
+----------------------------------------------------+--+
| CREATE TABLE `escape_comments_tbl1`( |
| `col1` string COMMENT 'a\nb') |
| COMMENT 'a\nb' |
| PARTITIONED BY ( |
| `p1` string COMMENT 'a\nb') |
| ROW FORMAT SERDE |
| 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' |
| STORED AS INPUTFORMAT |
| 'org.apache.hadoop.mapred.TextInputFormat' |
| OUTPUTFORMAT |
| 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' |
| LOCATION |
| 'file:/Users/petervary/data/apache/hive/warehouse/escape_comments_tbl1' |
| TBLPROPERTIES ( |
| 'transient_lastDdlTime'='1467380451') |
+----------------------------------------------------+--+
{noformat}
{noformat:title=Cli original}
CREATE TABLE `escape_comments_tbl1`(
`col1` string COMMENT 'a
b')
COMMENT 'a
b'
PARTITIONED BY (
`p1` string COMMENT 'a
b')
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
'file:/Users/petervary/data/apache/hive/warehouse/escape_comments_tbl1'
TBLPROPERTIES (
'transient_lastDdlTime'='1467380451')
{noformat}
{noformat:title=Cli patch}
CREATE TABLE `escape_comments_tbl1`(
`col1` string COMMENT 'a\nb')
COMMENT 'a\nb'
PARTITIONED BY (
`p1` string COMMENT 'a\nb')
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
'file:/Users/petervary/data/apache/hive/warehouse/escape_comments_tbl1'
TBLPROPERTIES (
'transient_lastDdlTime'='1467380451')
{noformat}
show create table escape_comments_view1;
{noformat:title=Beeline original}
+----------------------------------------------------+--+
| createtab_stmt |
+----------------------------------------------------+--+
| CREATE VIEW `escape_comments_view1` AS SELECT `col1` AS `col1` FROM (select
`escape_comments_tbl1`.`col1` from `default`.`escape_comments_tbl1`)
`default.escape_comments_view1` |
+----------------------------------------------------+--+
{noformat}
{noformat:title=Beeline patch}
+----------------------------------------------------+--+
| createtab_stmt |
+----------------------------------------------------+--+
| CREATE VIEW `escape_comments_view1` AS SELECT `col1` AS `col1` FROM (select
`escape_comments_tbl1`.`col1` from `default`.`escape_comments_tbl1`)
`default.escape_comments_view1` |
+----------------------------------------------------+--+
{noformat}
{noformat:title=Cli original}
CREATE VIEW `escape_comments_view1` AS SELECT `col1` AS `col1` FROM (select
`escape_comments_tbl1`.`col1` from `default`.`escape_comments_tbl1`)
`default.escape_comments_view1`
{noformat}
{noformat:title=Cli patch}
CREATE VIEW `escape_comments_view1` AS SELECT `col1` AS `col1` FROM (select
`escape_comments_tbl1`.`col1` from `default`.`escape_comments_tbl1`)
`default.escape_comments_view1`
{noformat}
show formatted index on escape_comments_tbl1;
{noformat:title=Beeline original}
+-----------------------+-----------------------+-----------------------+-----------------------------------------+-----------------------+-----------------------+--+
| idx_name | tab_name | col_names |
idx_tab_name | idx_type | comment
|
+-----------------------+-----------------------+-----------------------+-----------------------------------------+-----------------------+-----------------------+--+
| idx_name | tab_name | col_names |
idx_tab_name | idx_type | comment
|
| | NULL | NULL | NULL
| NULL | NULL
|
| | NULL | NULL | NULL
| NULL | NULL
|
| index2 | escape_comments_tbl1 | col1 |
default__escape_comments_tbl1_index2__ | compact | a
|
| b | | NULL | NULL
| NULL | NULL
|
+-----------------------+-----------------------+-----------------------+-----------------------------------------+-----------------------+-----------------------+--+
{noformat}
{noformat:title=Beeline patch}
+-----------------------+-----------------------+-----------------------+-----------------------------------------+-----------------------+-----------------------+--+
| idx_name | tab_name | col_names |
idx_tab_name | idx_type | comment
|
+-----------------------+-----------------------+-----------------------+-----------------------------------------+-----------------------+-----------------------+--+
| idx_name | tab_name | col_names |
idx_tab_name | idx_type | comment
|
| | NULL | NULL | NULL
| NULL | NULL
|
| | NULL | NULL | NULL
| NULL | NULL
|
| index2 | escape_comments_tbl1 | col1 |
default__escape_comments_tbl1_index2__ | compact | a\nb
|
+-----------------------+-----------------------+-----------------------+-----------------------------------------+-----------------------+-----------------------+--+
{noformat}
{noformat:title=Cli original}
idx_name tab_name col_names
idx_tab_name idx_type comment
index2 escape_comments_tbl1 col1
default__escape_comments_tbl1_index2__ compact a
b
{noformat}
{noformat:title=Cli patch}
idx_name tab_name col_names
idx_tab_name idx_type comment
index2 escape_comments_tbl1 col1
default__escape_comments_tbl1_index2__ compact a
b
{noformat}
> Column comments with "\n" character "corrupts" table metadata
> -------------------------------------------------------------
>
> Key: HIVE-14146
> URL: https://issues.apache.org/jira/browse/HIVE-14146
> Project: Hive
> Issue Type: Bug
> Components: Beeline
> Affects Versions: 2.2.0
> Reporter: Peter Vary
> Assignee: Peter Vary
> Attachments: HIVE-14146.10.patch, HIVE-14146.2.patch,
> HIVE-14146.3.patch, HIVE-14146.4.patch, HIVE-14146.5.patch,
> HIVE-14146.6.patch, HIVE-14146.7.patch, HIVE-14146.8.patch,
> HIVE-14146.9.patch, HIVE-14146.patch
>
>
> Create a table with the following(noting the \n in the COMMENT):
> {noformat}
> CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an
> individual’);
> {noformat}
> Describe shows that now the metadata is messed up:
> {noformat}
> beeline> describe commtest;
> +-------------------+------------+-----------------------+--+
> | col_name | data_type | comment |
> +-------------------+------------+-----------------------+--+
> | first_nm | string | Indicates First name |
> | of an individual | NULL | NULL |
> +-------------------+------------+-----------------------+--+
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
