[
https://issues.apache.org/jira/browse/IMPALA-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944016#comment-16944016
]
Sahil Takiar commented on IMPALA-7087:
--------------------------------------
[~ychena] I double checked, and it doesn't look like this issue is fixed.
If you run the tests in [https://gerrit.cloudera.org/#/c/12163/] against
master, you will see that they fail:
{code:java}
[localhost:21000] default> create table if not exists parquet_decimal_test
(small_dec decimal(38,2), med_dec decimal(38,2), large_dec decimal(38,2))
stored as parquet;
Query: create table if not exists parquet_decimal_test (small_dec
decimal(38,2), med_dec decimal(38,2), large_dec decimal(38,2)) stored as parquet
Table has been created.
Fetched 1 row(s) in 0.98s
hdfs dfs -copyFromLocal
testdata/data/binary_decimal_precision_and_scale_widening.parquet
/test-warehouse/parquet_decimal_test/
./bin/impala-shell.sh -B
[localhost:21000] default> invalidate metadata;
[localhost:21000] default> select * from parquet_decimal_test;
Query: select * from parquet_decimal_test
Query submitted at: 2019-10-03 13:49:51 (Coordinator:
http://impala-dev-env.vpc.cloudera.com:25000)
Query progress can be monitored at:
http://impala-dev-env.vpc.cloudera.com:25000/query_plan?query_id=0a4b5f748cd93687:4a4450c100000000
ERROR: File
'hdfs://localhost:20500/test-warehouse/parquet_decimal_test/binary_decimal_precision_and_scale_widening.parquet'
column 'small_dec' has an invalid type length. Expecting: 16 len in file:
4{code}
> Impala is unable to read Parquet decimal columns with lower precision/scale
> than table metadata
> -----------------------------------------------------------------------------------------------
>
> Key: IMPALA-7087
> URL: https://issues.apache.org/jira/browse/IMPALA-7087
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Yongzhi Chen
> Priority: Major
> Labels: decimal, parquet
>
> This is similar to IMPALA-2515, except relates to a different precision/scale
> in the file metadata rather than just a mismatch in the bytes used to store
> the data. In a lot of cases we should be able to convert the decimal type on
> the fly to the higher-precision type.
> {noformat}
> ERROR: File '/hdfs/path/000000_0_x_2' column 'alterd_decimal' has an invalid
> type length. Expecting: 11 len in file: 8
> {noformat}
> It would be convenient to allow reading parquet files where the
> precision/scale in the file can be converted to the precision/scale in the
> table metadata without loss of precision.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
