[jira] [Commented] (HIVE-3777) add a property in the partition to figure out if stats are accurate
[ https://issues.apache.org/jira/browse/HIVE-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13806561#comment-13806561 ] Dilip Joseph commented on HIVE-3777: [~ashutoshc] : I am not working on it. Please take over. > add a property in the partition to figure out if stats are accurate > --- > > Key: HIVE-3777 > URL: https://issues.apache.org/jira/browse/HIVE-3777 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Dilip Joseph > > Currently, stats task tries to update the statistics in the table/partition > being updated after the table/partition is loaded. In case of a failure to > update these stats (due to the any reason), the operation either succeeds > (writing inaccurate stats) or fails depending on whether hive.stats.reliable > is set to true. This can be bad for applications who do not always care about > reliable stats, since the query may have taken a long time to execute and then > fail eventually. > Another property should be added to the partition: areStatsAccurate. If > hive.stats.reliable is > set to false, and stats could not be computed correctly, the operation would > still succeed, update the stats, but set areStatsAccurate to false. > If the application cares about accurate stats, it can be obtained in the > background. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-3777) add a property in the partition to figure out if stats are accurate
[ https://issues.apache.org/jira/browse/HIVE-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Joseph updated HIVE-3777: --- Assignee: Ashutosh Chauhan (was: Dilip Joseph) > add a property in the partition to figure out if stats are accurate > --- > > Key: HIVE-3777 > URL: https://issues.apache.org/jira/browse/HIVE-3777 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Ashutosh Chauhan > > Currently, stats task tries to update the statistics in the table/partition > being updated after the table/partition is loaded. In case of a failure to > update these stats (due to the any reason), the operation either succeeds > (writing inaccurate stats) or fails depending on whether hive.stats.reliable > is set to true. This can be bad for applications who do not always care about > reliable stats, since the query may have taken a long time to execute and then > fail eventually. > Another property should be added to the partition: areStatsAccurate. If > hive.stats.reliable is > set to false, and stats could not be computed correctly, the operation would > still succeed, update the stats, but set areStatsAccurate to false. > If the application cares about accurate stats, it can be obtained in the > background. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-4409) Prevent incompatible column type changes
[
https://issues.apache.org/jira/browse/HIVE-4409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dilip Joseph updated HIVE-4409:
---
Description:
If a user changes the type of an existing column of a partitioned table to an
incompatible type, subsequent accesses of old partitions will result in a
ClassCastException (see example below). We should prevent the user from making
incompatible type changes. This feature will be controlled by a new config
parameter.
Example:
CREATE TABLE test_table123 (a INT, b MAP) PARTITIONED BY (ds
STRING) STORED AS SEQUENCEFILE;
INSERT OVERWRITE TABLE test_table123 PARTITION(ds="foo1") SELECT 1, MAP("a1",
"b1") FROM src LIMIT 1;
SELECT * from test_table123 WHERE ds="foo1";
ALTER TABLE test_table123 REPLACE COLUMNS (a INT, b STRING);
SELECT * from test_table123 WHERE ds="foo1";
The last SELECT fails with the following exception:
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
java.io.IOException: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1406)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790)
at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
at
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_class_cast(TestCliDriver.java:108)
The ALTER TABLE statement is blocked if you set the following parameter,
introduced int the fix to this JIRA:
SET hive.metastore.disallow.incompatible.col.type.changes=true;
was:
If a user changes the type of an existing column of a partitioned table to an
incompatible type, subsequent accesses of old partitions will result in a
ClassCastException (see example below). We should prevent the user from making
incompatible type changes. This feature will be controlled by a new config
parameter.
Example:
CREATE TABLE test_table123 (a INT, b MAP) PARTITIONED BY (ds
STRING) STORED AS SEQUENCEFILE;
INSERT OVERWRITE TABLE test_table123 PARTITION(ds="foo1") SELECT 1, MAP("a1",
"b1") FROM src LIMIT 1;
SELECT * from test_table123 WHERE ds="foo1";
SET hive.metastore.disallow.invalid.col.type.changes=true;
ALTER TABLE test_table123 REPLACE COLUMNS (a INT, b STRING);
SELECT * from test_table123 WHERE ds="foo1";
The last SELECT fails with the following exception:
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
java.io.IOException: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1406)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790)
at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
at
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_class_cast(TestCliDriver.java:108)
> Prevent incompatible column type changes
>
>
> Key: HIVE-4409
> URL: https://issues.apache.org/jira/browse/HIVE-4409
> Project: Hive
> Issue Type: Improvement
> Components: CLI, Metastore
>Affects Versions: 0.10.0
>Reporter: Dilip Joseph
>Assignee: Dilip Joseph
>Priority: Minor
> Fix Fo
[jira] [Updated] (HIVE-4409) Prevent incompatible column type changes
[
https://issues.apache.org/jira/browse/HIVE-4409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dilip Joseph updated HIVE-4409:
---
Status: Patch Available (was: Open)
> Prevent incompatible column type changes
>
>
> Key: HIVE-4409
> URL: https://issues.apache.org/jira/browse/HIVE-4409
> Project: Hive
> Issue Type: Improvement
> Components: CLI, Metastore
>Affects Versions: 0.10.0
>Reporter: Dilip Joseph
>Assignee: Dilip Joseph
>Priority: Minor
> Attachments: HIVE-4409.D10539.1.patch, HIVE-4409.D10539.2.patch
>
>
> If a user changes the type of an existing column of a partitioned table to an
> incompatible type, subsequent accesses of old partitions will result in a
> ClassCastException (see example below). We should prevent the user from
> making incompatible type changes. This feature will be controlled by a new
> config parameter.
> Example:
> CREATE TABLE test_table123 (a INT, b MAP) PARTITIONED BY (ds
> STRING) STORED AS SEQUENCEFILE;
> INSERT OVERWRITE TABLE test_table123 PARTITION(ds="foo1") SELECT 1, MAP("a1",
> "b1") FROM src LIMIT 1;
> SELECT * from test_table123 WHERE ds="foo1";
> SET hive.metastore.disallow.invalid.col.type.changes=true;
> ALTER TABLE test_table123 REPLACE COLUMNS (a INT, b STRING);
> SELECT * from test_table123 WHERE ds="foo1";
> The last SELECT fails with the following exception:
> Failed with exception java.io.IOException:java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
> java.io.IOException: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1406)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790)
> at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_class_cast(TestCliDriver.java:108)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4409) Prevent incompatible column type changes
[
https://issues.apache.org/jira/browse/HIVE-4409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dilip Joseph updated HIVE-4409:
---
Release Note: Added new config param
hive.metastore.disallow.incompatible.col.type.changes which is false by
default.
Status: Patch Available (was: Open)
> Prevent incompatible column type changes
>
>
> Key: HIVE-4409
> URL: https://issues.apache.org/jira/browse/HIVE-4409
> Project: Hive
> Issue Type: Improvement
> Components: CLI, Metastore
>Affects Versions: 0.10.0
>Reporter: Dilip Joseph
>Assignee: Dilip Joseph
>Priority: Minor
> Attachments: HIVE-4409.D10539.1.patch
>
>
> If a user changes the type of an existing column of a partitioned table to an
> incompatible type, subsequent accesses of old partitions will result in a
> ClassCastException (see example below). We should prevent the user from
> making incompatible type changes. This feature will be controlled by a new
> config parameter.
> Example:
> CREATE TABLE test_table123 (a INT, b MAP) PARTITIONED BY (ds
> STRING) STORED AS SEQUENCEFILE;
> INSERT OVERWRITE TABLE test_table123 PARTITION(ds="foo1") SELECT 1, MAP("a1",
> "b1") FROM src LIMIT 1;
> SELECT * from test_table123 WHERE ds="foo1";
> SET hive.metastore.disallow.invalid.col.type.changes=true;
> ALTER TABLE test_table123 REPLACE COLUMNS (a INT, b STRING);
> SELECT * from test_table123 WHERE ds="foo1";
> The last SELECT fails with the following exception:
> Failed with exception java.io.IOException:java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
> java.io.IOException: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1406)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790)
> at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_class_cast(TestCliDriver.java:108)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4409) Prevent incompatible column type changes
Dilip Joseph created HIVE-4409:
--
Summary: Prevent incompatible column type changes
Key: HIVE-4409
URL: https://issues.apache.org/jira/browse/HIVE-4409
Project: Hive
Issue Type: Improvement
Components: CLI, Metastore
Affects Versions: 0.10.0
Reporter: Dilip Joseph
Assignee: Dilip Joseph
Priority: Minor
If a user changes the type of an existing column of a partitioned table to an
incompatible type, subsequent accesses of old partitions will result in a
ClassCastException (see example below). We should prevent the user from making
incompatible type changes. This feature will be controlled by a new config
parameter.
Example:
CREATE TABLE test_table123 (a INT, b MAP) PARTITIONED BY (ds
STRING) STORED AS SEQUENCEFILE;
INSERT OVERWRITE TABLE test_table123 PARTITION(ds="foo1") SELECT 1, MAP("a1",
"b1") FROM src LIMIT 1;
SELECT * from test_table123 WHERE ds="foo1";
SET hive.metastore.disallow.invalid.col.type.changes=true;
ALTER TABLE test_table123 REPLACE COLUMNS (a INT, b STRING);
SELECT * from test_table123 WHERE ds="foo1";
The last SELECT fails with the following exception:
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
java.io.IOException: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1406)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790)
at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
at
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_class_cast(TestCliDriver.java:108)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3938) Hive MetaStore should send a single AddPartitionEvent for atomically added partition-set.
[ https://issues.apache.org/jira/browse/HIVE-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13606112#comment-13606112 ] Dilip Joseph commented on HIVE-3938: [~mithun] I can review it. Could you please upload another review and send the URL. Thanks. > Hive MetaStore should send a single AddPartitionEvent for atomically added > partition-set. > - > > Key: HIVE-3938 > URL: https://issues.apache.org/jira/browse/HIVE-3938 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.10.0 >Reporter: Mithun Radhakrishnan >Assignee: Mithun Radhakrishnan > Attachments: HIVE-3938.trunk.patch > > > HiveMetaStore::add_partitions() currently adds all partitions specified in > one call using a single meta-store transaction. This acts correctly. However, > there's one AddPartitionEvent created per partition specified. > Ideally, the set of partitions added atomically can be communicated using a > single AddPartitionEvent, such that they are consumed together. > I'll post a patch that does this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4004) Incorrect status for AddPartition metastore event if RawStore commit fails
[ https://issues.apache.org/jira/browse/HIVE-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574898#comment-13574898 ] Dilip Joseph commented on HIVE-4004: Code review at https://reviews.facebook.net/D8481 > Incorrect status for AddPartition metastore event if RawStore commit fails > -- > > Key: HIVE-4004 > URL: https://issues.apache.org/jira/browse/HIVE-4004 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.10.0 >Reporter: Dilip Joseph >Assignee: Dilip Joseph >Priority: Minor > Fix For: 0.11.0 > > Attachments: HIVE-4004.1.patch.txt > > > For ADD PARTITION operations, the AddPartitionEvent does not care if the > RawStore commit succeeded or not. This means that an AddPartitionEvent with > status=true is fired even if the the actual ADD PARTITION operation failed. > This will confuse any AddPartitionEvent listeners. > Other MetastoreListenerEvents like CreateTableEvent correctly incorporate the > status of the RawStore commit. Only AddPartitionEvent has this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4004) Incorrect status for AddPartition metastore event if RawStore commit fails
[ https://issues.apache.org/jira/browse/HIVE-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Joseph updated HIVE-4004: --- Fix Version/s: 0.11.0 Status: Patch Available (was: Open) > Incorrect status for AddPartition metastore event if RawStore commit fails > -- > > Key: HIVE-4004 > URL: https://issues.apache.org/jira/browse/HIVE-4004 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.10.0 >Reporter: Dilip Joseph >Assignee: Dilip Joseph >Priority: Minor > Fix For: 0.11.0 > > Attachments: HIVE-4004.1.patch.txt > > > For ADD PARTITION operations, the AddPartitionEvent does not care if the > RawStore commit succeeded or not. This means that an AddPartitionEvent with > status=true is fired even if the the actual ADD PARTITION operation failed. > This will confuse any AddPartitionEvent listeners. > Other MetastoreListenerEvents like CreateTableEvent correctly incorporate the > status of the RawStore commit. Only AddPartitionEvent has this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4004) Incorrect status for AddPartition metastore event if RawStore commit fails
[ https://issues.apache.org/jira/browse/HIVE-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Joseph updated HIVE-4004: --- Attachment: HIVE-4004.1.patch.txt > Incorrect status for AddPartition metastore event if RawStore commit fails > -- > > Key: HIVE-4004 > URL: https://issues.apache.org/jira/browse/HIVE-4004 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.10.0 >Reporter: Dilip Joseph >Assignee: Dilip Joseph >Priority: Minor > Attachments: HIVE-4004.1.patch.txt > > > For ADD PARTITION operations, the AddPartitionEvent does not care if the > RawStore commit succeeded or not. This means that an AddPartitionEvent with > status=true is fired even if the the actual ADD PARTITION operation failed. > This will confuse any AddPartitionEvent listeners. > Other MetastoreListenerEvents like CreateTableEvent correctly incorporate the > status of the RawStore commit. Only AddPartitionEvent has this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4004) Incorrect status for AddPartition metastore event if RawStore commit fails
Dilip Joseph created HIVE-4004: -- Summary: Incorrect status for AddPartition metastore event if RawStore commit fails Key: HIVE-4004 URL: https://issues.apache.org/jira/browse/HIVE-4004 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Dilip Joseph Assignee: Dilip Joseph Priority: Minor For ADD PARTITION operations, the AddPartitionEvent does not care if the RawStore commit succeeded or not. This means that an AddPartitionEvent with status=true is fired even if the the actual ADD PARTITION operation failed. This will confuse any AddPartitionEvent listeners. Other MetastoreListenerEvents like CreateTableEvent correctly incorporate the status of the RawStore commit. Only AddPartitionEvent has this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3884) Better align columns in DESCRIBE table_name output to make more human-readable
[ https://issues.apache.org/jira/browse/HIVE-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Joseph updated HIVE-3884: --- Status: Patch Available (was: Open) > Better align columns in DESCRIBE table_name output to make more human-readable > -- > > Key: HIVE-3884 > URL: https://issues.apache.org/jira/browse/HIVE-3884 > Project: Hive > Issue Type: Improvement > Components: CLI >Affects Versions: 0.9.0 >Reporter: Dilip Joseph >Assignee: Dilip Joseph >Priority: Minor > Fix For: 0.11.0 > > Attachments: describe_test_table.png, HIVE-3884.1.patch.txt, > HIVE-3884.2.patch.txt > > > If a table contains very long comments or very long column names, the output > of DESCRIBE table_name is not aligned nicely. The attached screenshot shows > the following two problems: > 1. Rows with long column names do not align well with other columns. > 2. Rows with long comments wrap to the next line, and make it hard to read > the output. The wrapping behavior depends on the width of the user's > terminal width. > It will be nice to have a DESCRIBE PRETTY table_name command that will > produce nicely formatted output that avoids the two problems mentioned above. > It is better to introduce a new DESCRIBE PRETTY command rather than change > the behavior of the existing DESCRIBE or DESCRIBE FORMATTED commands, so that > we avoid breaking any scripts that automatically parse the output. > Since the pretty formatting depends on the current terminal width, we need a > new hive conf parameter to tell the CLI to auto-detect the current terminal > width or to use a fixed width (needed for unit tests). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3884) Better align columns in DESCRIBE table_name output to make more human-readable
[ https://issues.apache.org/jira/browse/HIVE-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Joseph updated HIVE-3884: --- Attachment: HIVE-3884.2.patch.txt Added comment about tabs at end of comment output. Minor refactoring. > Better align columns in DESCRIBE table_name output to make more human-readable > -- > > Key: HIVE-3884 > URL: https://issues.apache.org/jira/browse/HIVE-3884 > Project: Hive > Issue Type: Improvement > Components: CLI >Affects Versions: 0.9.0 >Reporter: Dilip Joseph >Assignee: Dilip Joseph >Priority: Minor > Fix For: 0.11.0 > > Attachments: describe_test_table.png, HIVE-3884.1.patch.txt, > HIVE-3884.2.patch.txt > > > If a table contains very long comments or very long column names, the output > of DESCRIBE table_name is not aligned nicely. The attached screenshot shows > the following two problems: > 1. Rows with long column names do not align well with other columns. > 2. Rows with long comments wrap to the next line, and make it hard to read > the output. The wrapping behavior depends on the width of the user's > terminal width. > It will be nice to have a DESCRIBE PRETTY table_name command that will > produce nicely formatted output that avoids the two problems mentioned above. > It is better to introduce a new DESCRIBE PRETTY command rather than change > the behavior of the existing DESCRIBE or DESCRIBE FORMATTED commands, so that > we avoid breaking any scripts that automatically parse the output. > Since the pretty formatting depends on the current terminal width, we need a > new hive conf parameter to tell the CLI to auto-detect the current terminal > width or to use a fixed width (needed for unit tests). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3884) Better align columns in DESCRIBE table_name output to make more human-readable
[ https://issues.apache.org/jira/browse/HIVE-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Joseph updated HIVE-3884: --- Fix Version/s: 0.11.0 Release Note: Introduces new DESCRIBE PRETTY table_name command and hive.cli.pretty.output.num.cols conf parameter. Status: Patch Available (was: Open) Phabricator diff is at https://reviews.facebook.net/D7851 > Better align columns in DESCRIBE table_name output to make more human-readable > -- > > Key: HIVE-3884 > URL: https://issues.apache.org/jira/browse/HIVE-3884 > Project: Hive > Issue Type: Improvement > Components: CLI >Affects Versions: 0.9.0 >Reporter: Dilip Joseph >Assignee: Dilip Joseph >Priority: Minor > Fix For: 0.11.0 > > Attachments: describe_test_table.png, HIVE-3884.1.patch.txt > > > If a table contains very long comments or very long column names, the output > of DESCRIBE table_name is not aligned nicely. The attached screenshot shows > the following two problems: > 1. Rows with long column names do not align well with other columns. > 2. Rows with long comments wrap to the next line, and make it hard to read > the output. The wrapping behavior depends on the width of the user's > terminal width. > It will be nice to have a DESCRIBE PRETTY table_name command that will > produce nicely formatted output that avoids the two problems mentioned above. > It is better to introduce a new DESCRIBE PRETTY command rather than change > the behavior of the existing DESCRIBE or DESCRIBE FORMATTED commands, so that > we avoid breaking any scripts that automatically parse the output. > Since the pretty formatting depends on the current terminal width, we need a > new hive conf parameter to tell the CLI to auto-detect the current terminal > width or to use a fixed width (needed for unit tests). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3884) Better align columns in DESCRIBE table_name output to make more human-readable
[ https://issues.apache.org/jira/browse/HIVE-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Joseph updated HIVE-3884: --- Attachment: HIVE-3884.1.patch.txt > Better align columns in DESCRIBE table_name output to make more human-readable > -- > > Key: HIVE-3884 > URL: https://issues.apache.org/jira/browse/HIVE-3884 > Project: Hive > Issue Type: Improvement > Components: CLI >Affects Versions: 0.9.0 >Reporter: Dilip Joseph >Assignee: Dilip Joseph >Priority: Minor > Attachments: describe_test_table.png, HIVE-3884.1.patch.txt > > > If a table contains very long comments or very long column names, the output > of DESCRIBE table_name is not aligned nicely. The attached screenshot shows > the following two problems: > 1. Rows with long column names do not align well with other columns. > 2. Rows with long comments wrap to the next line, and make it hard to read > the output. The wrapping behavior depends on the width of the user's > terminal width. > It will be nice to have a DESCRIBE PRETTY table_name command that will > produce nicely formatted output that avoids the two problems mentioned above. > It is better to introduce a new DESCRIBE PRETTY command rather than change > the behavior of the existing DESCRIBE or DESCRIBE FORMATTED commands, so that > we avoid breaking any scripts that automatically parse the output. > Since the pretty formatting depends on the current terminal width, we need a > new hive conf parameter to tell the CLI to auto-detect the current terminal > width or to use a fixed width (needed for unit tests). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3884) Better align columns in DESCRIBE table_name output to make more human-readable
[ https://issues.apache.org/jira/browse/HIVE-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Joseph reassigned HIVE-3884: -- Assignee: Dilip Joseph (was: Gang Tim Liu) > Better align columns in DESCRIBE table_name output to make more human-readable > -- > > Key: HIVE-3884 > URL: https://issues.apache.org/jira/browse/HIVE-3884 > Project: Hive > Issue Type: Improvement > Components: CLI >Affects Versions: 0.9.0 >Reporter: Dilip Joseph >Assignee: Dilip Joseph >Priority: Minor > Attachments: describe_test_table.png > > > If a table contains very long comments or very long column names, the output > of DESCRIBE table_name is not aligned nicely. The attached screenshot shows > the following two problems: > 1. Rows with long column names do not align well with other columns. > 2. Rows with long comments wrap to the next line, and make it hard to read > the output. The wrapping behavior depends on the width of the user's > terminal width. > It will be nice to have a DESCRIBE PRETTY table_name command that will > produce nicely formatted output that avoids the two problems mentioned above. > It is better to introduce a new DESCRIBE PRETTY command rather than change > the behavior of the existing DESCRIBE or DESCRIBE FORMATTED commands, so that > we avoid breaking any scripts that automatically parse the output. > Since the pretty formatting depends on the current terminal width, we need a > new hive conf parameter to tell the CLI to auto-detect the current terminal > width or to use a fixed width (needed for unit tests). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3884) Better align columns in DESCRIBE table_name output to make more human-readable
[ https://issues.apache.org/jira/browse/HIVE-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Joseph updated HIVE-3884: --- Attachment: describe_test_table.png Output of DESCRIBE test_table for the following table: CREATE TABLE test_table( col1 INT COMMENT 'col1 one line comment', col2 STRING COMMENT 'col2 very long comment that is greater than 80 chars and is likely to spill into multiple lines', col3 STRING COMMENT 'col3 very long multi-line comment where each line is very long by itself and is likely to spill into multiple lines. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin in dolor nisl, sodales adipiscing tortor. Integer venenatis', col4 INT COMMENT 'col4 one line comment', col5_VeryLongColumnNameThatMessesUpAlignment INT COMMENT 'col5 one line comment', col6 STRING COMMENT 'col6 one line comment' ); > Better align columns in DESCRIBE table_name output to make more human-readable > -- > > Key: HIVE-3884 > URL: https://issues.apache.org/jira/browse/HIVE-3884 > Project: Hive > Issue Type: Improvement > Components: CLI >Affects Versions: 0.9.0 >Reporter: Dilip Joseph >Priority: Minor > Attachments: describe_test_table.png > > > If a table contains very long comments or very long column names, the output > of DESCRIBE table_name is not aligned nicely. The attached screenshot shows > the following two problems: > 1. Rows with long column names do not align well with other columns. > 2. Rows with long comments wrap to the next line, and make it hard to read > the output. The wrapping behavior depends on the width of the user's > terminal width. > It will be nice to have a DESCRIBE PRETTY table_name command that will > produce nicely formatted output that avoids the two problems mentioned above. > It is better to introduce a new DESCRIBE PRETTY command rather than change > the behavior of the existing DESCRIBE or DESCRIBE FORMATTED commands, so that > we avoid breaking any scripts that automatically parse the output. > Since the pretty formatting depends on the current terminal width, we need a > new hive conf parameter to tell the CLI to auto-detect the current terminal > width or to use a fixed width (needed for unit tests). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3884) Better align columns in DESCRIBE table_name output to make more human-readable
Dilip Joseph created HIVE-3884: -- Summary: Better align columns in DESCRIBE table_name output to make more human-readable Key: HIVE-3884 URL: https://issues.apache.org/jira/browse/HIVE-3884 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Dilip Joseph Priority: Minor If a table contains very long comments or very long column names, the output of DESCRIBE table_name is not aligned nicely. The attached screenshot shows the following two problems: 1. Rows with long column names do not align well with other columns. 2. Rows with long comments wrap to the next line, and make it hard to read the output. The wrapping behavior depends on the width of the user's terminal width. It will be nice to have a DESCRIBE PRETTY table_name command that will produce nicely formatted output that avoids the two problems mentioned above. It is better to introduce a new DESCRIBE PRETTY command rather than change the behavior of the existing DESCRIBE or DESCRIBE FORMATTED commands, so that we avoid breaking any scripts that automatically parse the output. Since the pretty formatting depends on the current terminal width, we need a new hive conf parameter to tell the CLI to auto-detect the current terminal width or to use a fixed width (needed for unit tests). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
