[jira] [Commented] (HIVE-14085) Allow type widening primitive conversion on hive/parquet tables
[
https://issues.apache.org/jira/browse/HIVE-14085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351677#comment-15351677
]
Sergio Peña commented on HIVE-14085:
Patch looks good, and tests are not related. The one that fails is a flaky
issue detected in older tests.
+1
> Allow type widening primitive conversion on hive/parquet tables
> ---
>
> Key: HIVE-14085
> URL: https://issues.apache.org/jira/browse/HIVE-14085
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-14085.1.patch
>
>
> There is a JIRA ticket on upstream that brought this usability improvement in
> Hive to support auto type widening for Parquet tables. See
> https://issues.apache.org/jira/browse/HIVE-12080
> This improvement is very useful for users who have schema evolution on their
> tables. For example, a Hive table with a "bigint" can read parquet files with
> "int32" and "int64" types.
> The patch only supports widening conversions from int->bigint and
> float->double. We should support more types to allow users read their changed
> parquet schema.
> Here's a list of widening conversions we should support:
> {code}
> tinyint -> smallint,int,bigint,float,double
> smallint -> int,bigint,float,double
> int -> bigint,float,double
> bigint -> float,double
> float -> double
> double -> --
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14085) Allow type widening primitive conversion on hive/parquet tables
[
https://issues.apache.org/jira/browse/HIVE-14085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351534#comment-15351534
]
Vihang Karajgaonkar commented on HIVE-14085:
All the tests except for TestMiniSparkOnYarnCliDriver for index_bitmap3.q are
failing even without this change in the latest code base in master branch.
index_bitmap3.q with TestMiniSparkOnYarnCliDriver is working fine locally so
its failure doesn't seem to be related to this change.
> Allow type widening primitive conversion on hive/parquet tables
> ---
>
> Key: HIVE-14085
> URL: https://issues.apache.org/jira/browse/HIVE-14085
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-14085.1.patch
>
>
> There is a JIRA ticket on upstream that brought this usability improvement in
> Hive to support auto type widening for Parquet tables. See
> https://issues.apache.org/jira/browse/HIVE-12080
> This improvement is very useful for users who have schema evolution on their
> tables. For example, a Hive table with a "bigint" can read parquet files with
> "int32" and "int64" types.
> The patch only supports widening conversions from int->bigint and
> float->double. We should support more types to allow users read their changed
> parquet schema.
> Here's a list of widening conversions we should support:
> {code}
> tinyint -> smallint,int,bigint,float,double
> smallint -> int,bigint,float,double
> int -> bigint,float,double
> bigint -> float,double
> float -> double
> double -> --
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14085) Allow type widening primitive conversion on hive/parquet tables
[
https://issues.apache.org/jira/browse/HIVE-14085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348973#comment-15348973
]
Hive QA commented on HIVE-14085:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12813125/HIVE-14085.1.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10265 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/257/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/257/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-257/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12813125 - PreCommit-HIVE-MASTER-Build
> Allow type widening primitive conversion on hive/parquet tables
> ---
>
> Key: HIVE-14085
> URL: https://issues.apache.org/jira/browse/HIVE-14085
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-14085.1.patch
>
>
> There is a JIRA ticket on upstream that brought this usability improvement in
> Hive to support auto type widening for Parquet tables. See
> https://issues.apache.org/jira/browse/HIVE-12080
> This improvement is very useful for users who have schema evolution on their
> tables. For example, a Hive table with a "bigint" can read parquet files with
> "int32" and "int64" types.
> The patch only supports widening conversions from int->bigint and
> float->double. We should support more types to allow users read their changed
> parquet schema.
> Here's a list of widening conversions we should support:
> {code}
> tinyint -> smallint,int,bigint,float,double
> smallint -> int,bigint,float,double
> int -> bigint,float,double
> bigint -> float,double
> float -> double
> double -> --
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14085) Allow type widening primitive conversion on hive/parquet tables
[
https://issues.apache.org/jira/browse/HIVE-14085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348681#comment-15348681
]
Vihang Karajgaonkar commented on HIVE-14085:
Review board link : https://reviews.apache.org/r/49215
> Allow type widening primitive conversion on hive/parquet tables
> ---
>
> Key: HIVE-14085
> URL: https://issues.apache.org/jira/browse/HIVE-14085
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-14085.1.patch
>
>
> There is a JIRA ticket on upstream that brought this usability improvement in
> Hive to support auto type widening for Parquet tables. See
> https://issues.apache.org/jira/browse/HIVE-12080
> This improvement is very useful for users who have schema evolution on their
> tables. For example, a Hive table with a "bigint" can read parquet files with
> "int32" and "int64" types.
> The patch only supports widening conversions from int->bigint and
> float->double. We should support more types to allow users read their changed
> parquet schema.
> Here's a list of widening conversions we should support:
> {code}
> tinyint -> smallint,int,bigint,float,double
> smallint -> int,bigint,float,double
> int -> bigint,float,double
> bigint -> float,double
> float -> double
> double -> --
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14085) Allow type widening primitive conversion on hive/parquet tables
[
https://issues.apache.org/jira/browse/HIVE-14085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348665#comment-15348665
]
Vihang Karajgaonkar commented on HIVE-14085:
[~spena] Can you take a look at the patch? Thanks!
> Allow type widening primitive conversion on hive/parquet tables
> ---
>
> Key: HIVE-14085
> URL: https://issues.apache.org/jira/browse/HIVE-14085
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-14085.1.patch
>
>
> There is a JIRA ticket on upstream that brought this usability improvement in
> Hive to support auto type widening for Parquet tables. See
> https://issues.apache.org/jira/browse/HIVE-12080
> This improvement is very useful for users who have schema evolution on their
> tables. For example, a Hive table with a "bigint" can read parquet files with
> "int32" and "int64" types.
> The patch only supports widening conversions from int->bigint and
> float->double. We should support more types to allow users read their changed
> parquet schema.
> Here's a list of widening conversions we should support:
> {code}
> tinyint -> smallint,int,bigint,float,double
> smallint -> int,bigint,float,double
> int -> bigint,float,double
> bigint -> float,double
> float -> double
> double -> --
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
