[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044325#comment-14044325 ] Lefty Leverenz commented on HIVE-6367: -- Documented in the wiki, please review and remove TODOC14 label if it's okay: * [Language Manual -- Parquet -- Limitations | https://cwiki.apache.org/confluence/display/Hive/Parquet#Parquet-Limitations] Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Xuefu Zhang Labels: Parquet, TODOC14 Fix For: 0.14.0 Attachments: HIVE-6367.patch, dec.parq Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044329#comment-14044329 ] Xuefu Zhang commented on HIVE-6367: --- Thanks, [~leftylev]. I removed the label. Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Xuefu Zhang Labels: Parquet Fix For: 0.14.0 Attachments: HIVE-6367.patch, dec.parq Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010722#comment-14010722 ] Navis commented on HIVE-6367: - [~xuefuz] [~brocknoland] dec_comp.txt is missing, making parquet_decimal1 fail. Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Xuefu Zhang Labels: Parquet Fix For: 0.14.0 Attachments: HIVE-6367.patch, dec.parq Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010727#comment-14010727 ] Xuefu Zhang commented on HIVE-6367: --- It's addressed in the followiup jira, HIVE-7123. Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Xuefu Zhang Labels: Parquet Fix For: 0.14.0 Attachments: HIVE-6367.patch, dec.parq Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010775#comment-14010775 ] Navis commented on HIVE-6367: - [~xuefuz] Ah, thanks. Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Xuefu Zhang Labels: Parquet Fix For: 0.14.0 Attachments: HIVE-6367.patch, dec.parq Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008166#comment-14008166 ] Brock Noland commented on HIVE-6367: +1 Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Xuefu Zhang Labels: Parquet Attachments: HIVE-6367.patch, dec.parq Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008274#comment-14008274 ] Lefty Leverenz commented on HIVE-6367: -- This needs to be documented in the wiki with a 0.14.0 note: * [Parquet - Limitations | https://cwiki.apache.org/confluence/display/Hive/Parquet#Parquet-Limitations] ** Binary, timestamp, date, char, varchar or decimal support are pending (HIVE-6384) But the doc should wait until 0.14.0 is released, so please add a release note as a reminder. Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Xuefu Zhang Labels: Parquet Fix For: 0.14.0 Attachments: HIVE-6367.patch, dec.parq Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[
https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007915#comment-14007915
]
Hive QA commented on HIVE-6367:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646583/HIVE-6367.patch
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 5537 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/274/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/274/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-274/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12646583
Implement Decimal in ParquetSerde
-
Key: HIVE-6367
URL: https://issues.apache.org/jira/browse/HIVE-6367
Project: Hive
Issue Type: Sub-task
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Brock Noland
Assignee: Xuefu Zhang
Labels: Parquet
Attachments: HIVE-6367.patch, dec.parq
Some code in the Parquet Serde deals with decimal and other does not. For
example in ETypeConverter we convert Decimal to double (which is invalid)
whereas in DataWritableWriter and other locations we throw an exception if
decimal is used.
This JIRA is to implement decimal support.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13893436#comment-13893436 ] Xuefu Zhang commented on HIVE-6367: --- Parquet doesn't have native decimal type. However, it's possible to support Hive decimal data by serializing it into one of Parquet supported data type such as binary. This is the approach that's going to move forward. Comments and feedback are greatly appreciated. Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Task Reporter: Brock Noland Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13893524#comment-13893524 ] Brock Noland commented on HIVE-6367: FYI I converted this to a subtask of HIVE-6384 so I could link a single JIRA in the docs. Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Reporter: Brock Noland Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
