[jira] [Commented] (HIVE-12320) hive.metastore.disallow.incompatible.col.type.changes should be true by default
[ https://issues.apache.org/jira/browse/HIVE-12320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14993329#comment-14993329 ] Lefty Leverenz commented on HIVE-12320: --- Doc note: *hive.metastore.disallow.incompatible.col.type.changes* was introduced by HIVE-4409 in release 0.12.0. It needs to be documented in the wiki along with this change of default value in 2.0.0. * [Configuration Properties -- MetaStore | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-MetaStore] > hive.metastore.disallow.incompatible.col.type.changes should be true by > default > --- > > Key: HIVE-12320 > URL: https://issues.apache.org/jira/browse/HIVE-12320 > Project: Hive > Issue Type: Improvement > Components: Configuration, Types >Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Labels: TODOC2.0 > Fix For: 2.0.0 > > Attachments: HIVE-12320.2.patch, HIVE-12320.3.patch, > HIVE-12320.5.patch, HIVE-12320.patch > > > By default all types of schema changes are permitted. This config adds > capability to disallow incompatible column type changes. This should be on by > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12320) hive.metastore.disallow.incompatible.col.type.changes should be true by default
[ https://issues.apache.org/jira/browse/HIVE-12320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992204#comment-14992204 ] Ashutosh Chauhan commented on HIVE-12320: - Yeah.. can do that at time of check in, if remaining parts of patch looks alright. > hive.metastore.disallow.incompatible.col.type.changes should be true by > default > --- > > Key: HIVE-12320 > URL: https://issues.apache.org/jira/browse/HIVE-12320 > Project: Hive > Issue Type: Improvement > Components: Configuration, Types >Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12320.2.patch, HIVE-12320.3.patch, > HIVE-12320.5.patch, HIVE-12320.patch > > > By default all types of schema changes are permitted. This config adds > capability to disallow incompatible column type changes. This should be on by > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12320) hive.metastore.disallow.incompatible.col.type.changes should be true by default
[ https://issues.apache.org/jira/browse/HIVE-12320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992196#comment-14992196 ] Jason Dere commented on HIVE-12320: --- I think lineage3.q.out was also recently updated by HIVE-12317 and needs to be regenerated again for this patch. > hive.metastore.disallow.incompatible.col.type.changes should be true by > default > --- > > Key: HIVE-12320 > URL: https://issues.apache.org/jira/browse/HIVE-12320 > Project: Hive > Issue Type: Improvement > Components: Configuration, Types >Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12320.2.patch, HIVE-12320.3.patch, > HIVE-12320.5.patch, HIVE-12320.patch > > > By default all types of schema changes are permitted. This config adds > capability to disallow incompatible column type changes. This should be on by > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12320) hive.metastore.disallow.incompatible.col.type.changes should be true by default
[
https://issues.apache.org/jira/browse/HIVE-12320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990732#comment-14990732
]
Hive QA commented on HIVE-12320:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12770628/HIVE-12320.5.patch
{color:green}SUCCESS:{color} +1 due to 17 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 9768 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lineage3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_tez_union_with_udf
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_with_udf
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5922/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5922/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5922/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12770628 - PreCommit-HIVE-TRUNK-Build
> hive.metastore.disallow.incompatible.col.type.changes should be true by
> default
> ---
>
> Key: HIVE-12320
> URL: https://issues.apache.org/jira/browse/HIVE-12320
> Project: Hive
> Issue Type: Improvement
> Components: Configuration, Types
>Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-12320.2.patch, HIVE-12320.3.patch,
> HIVE-12320.5.patch, HIVE-12320.patch
>
>
> By default all types of schema changes are permitted. This config adds
> capability to disallow incompatible column type changes. This should be on by
> default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12320) hive.metastore.disallow.incompatible.col.type.changes should be true by default
[
https://issues.apache.org/jira/browse/HIVE-12320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14989197#comment-14989197
]
Hive QA commented on HIVE-12320:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12770416/HIVE-12320.3.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 9768 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnarserde_create_shortcut
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lineage3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat16
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rename_column
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_tez_union_with_udf
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_with_udf
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_disallow_incompatible_type_change_on1
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.hcatalog.api.TestHCatClient.testUpdateTableSchema
org.apache.hive.hcatalog.cli.TestSemanticAnalysis.testAddReplaceCols
org.apache.hive.hcatalog.cli.TestSemanticAnalysis.testChangeColumns
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5913/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5913/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5913/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12770416 - PreCommit-HIVE-TRUNK-Build
> hive.metastore.disallow.incompatible.col.type.changes should be true by
> default
> ---
>
> Key: HIVE-12320
> URL: https://issues.apache.org/jira/browse/HIVE-12320
> Project: Hive
> Issue Type: Improvement
> Components: Configuration, Types
>Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-12320.2.patch, HIVE-12320.3.patch, HIVE-12320.patch
>
>
> By default all types of schema changes are permitted. This config adds
> capability to disallow incompatible column type changes. This should be on by
> default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12320) hive.metastore.disallow.incompatible.col.type.changes should be true by default
[
https://issues.apache.org/jira/browse/HIVE-12320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987126#comment-14987126
]
Hive QA commented on HIVE-12320:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12770217/HIVE-12320.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 9760 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_char1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_char2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_change_col
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_table_cascade
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_varchar1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_varchar2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_partitioned
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_int_type_promotion
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_schema_evolution
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_parquet_types
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5899/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5899/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5899/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12770217 - PreCommit-HIVE-TRUNK-Build
> hive.metastore.disallow.incompatible.col.type.changes should be true by
> default
> ---
>
> Key: HIVE-12320
> URL: https://issues.apache.org/jira/browse/HIVE-12320
> Project: Hive
> Issue Type: Improvement
> Components: Configuration, Metastore
>Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-12320.patch
>
>
> By default all types of schema changes are permitted. This config adds
> capability to disallow incompatible column type changes. This should be on by
> default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12320) hive.metastore.disallow.incompatible.col.type.changes should be true by default
[
https://issues.apache.org/jira/browse/HIVE-12320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988426#comment-14988426
]
Hive QA commented on HIVE-12320:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12770364/HIVE-12320.2.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 9766 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables_compact
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5906/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5906/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5906/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12770364 - PreCommit-HIVE-TRUNK-Build
> hive.metastore.disallow.incompatible.col.type.changes should be true by
> default
> ---
>
> Key: HIVE-12320
> URL: https://issues.apache.org/jira/browse/HIVE-12320
> Project: Hive
> Issue Type: Improvement
> Components: Configuration, Types
>Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-12320.2.patch, HIVE-12320.3.patch, HIVE-12320.patch
>
>
> By default all types of schema changes are permitted. This config adds
> capability to disallow incompatible column type changes. This should be on by
> default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12320) hive.metastore.disallow.incompatible.col.type.changes should be true by default
[ https://issues.apache.org/jira/browse/HIVE-12320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988533#comment-14988533 ] Jason Dere commented on HIVE-12320: --- I think this looks ok. One thing I will note is that it is possible to change a table column from short to decimal, but if you then try to change it back from decimal these rules will kick in and complain because decimal is not implicitly convertible to short. So this would require the user to set hive.metastore.disallow.incompatible.col.type.changes=false before doing this. > hive.metastore.disallow.incompatible.col.type.changes should be true by > default > --- > > Key: HIVE-12320 > URL: https://issues.apache.org/jira/browse/HIVE-12320 > Project: Hive > Issue Type: Improvement > Components: Configuration, Types >Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12320.2.patch, HIVE-12320.3.patch, HIVE-12320.patch > > > By default all types of schema changes are permitted. This config adds > capability to disallow incompatible column type changes. This should be on by > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12320) hive.metastore.disallow.incompatible.col.type.changes should be true by default
[ https://issues.apache.org/jira/browse/HIVE-12320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988599#comment-14988599 ] Ashutosh Chauhan commented on HIVE-12320: - Thats right and that indeed is intentional, since once you go to wider type going back to narrower type may give you wrong/corrupt/truncated data. Those are the cases which we want to prevent by default. But, if user still wants narrower type he can chose to set config to false to do it. > hive.metastore.disallow.incompatible.col.type.changes should be true by > default > --- > > Key: HIVE-12320 > URL: https://issues.apache.org/jira/browse/HIVE-12320 > Project: Hive > Issue Type: Improvement > Components: Configuration, Types >Affects Versions: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12320.2.patch, HIVE-12320.3.patch, HIVE-12320.patch > > > By default all types of schema changes are permitted. This config adds > capability to disallow incompatible column type changes. This should be on by > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
