[ https://issues.apache.org/jira/browse/IMPALA-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17384976#comment-17384976 ]
ASF subversion and git services commented on IMPALA-7087: --------------------------------------------------------- Commit b0d364cc5a62fcfb7ad26d52d3f492239151a3c1 in impala's branch refs/heads/master from stiga-huang [ https://gitbox.apache.org/repos/asf?p=impala.git;h=b0d364c ] IMPALA-10808: Fix crash on illegal decimal precision in parquet file schema In IMPALA-7087, we allow parquet file schema to have decimal precision lower than the one in table schema. However, we should check illegal precisions like 0 or negative values. They will hit a DCHECK in ColumnType::GetDecimalByteSize(int). Change-Id: If84fe14740aea97461ebccedffa4c2ca393333d8 Reviewed-on: http://gerrit.cloudera.org:8080/17701 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> > Impala is unable to read Parquet decimal columns with lower precision/scale > than table metadata > ----------------------------------------------------------------------------------------------- > > Key: IMPALA-7087 > URL: https://issues.apache.org/jira/browse/IMPALA-7087 > Project: IMPALA > Issue Type: Sub-task > Components: Backend > Reporter: Tim Armstrong > Assignee: Zoltán Borók-Nagy > Priority: Major > Labels: decimal, parquet, ramp-up > Fix For: Impala 4.1 > > Attachments: binary_decimal_precision_and_scale_widening.parquet > > > This is similar to IMPALA-2515, except relates to a different precision/scale > in the file metadata rather than just a mismatch in the bytes used to store > the data. In a lot of cases we should be able to convert the decimal type on > the fly to the higher-precision type. > {noformat} > ERROR: File '/hdfs/path/000000_0_x_2' column 'alterd_decimal' has an invalid > type length. Expecting: 11 len in file: 8 > {noformat} > It would be convenient to allow reading parquet files where the > precision/scale in the file can be converted to the precision/scale in the > table metadata without loss of precision. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org