[ https://issues.apache.org/jira/browse/IMPALA-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Onur Tokat updated IMPALA-2515: ------------------------------- Attachment: image-2020-02-07-11-33-27-944.png > Impala is unable to read a Parquet decimal column if size is larger than > needed > ------------------------------------------------------------------------------- > > Key: IMPALA-2515 > URL: https://issues.apache.org/jira/browse/IMPALA-2515 > Project: IMPALA > Issue Type: Sub-task > Components: Backend > Affects Versions: Impala 2.3.0 > Reporter: Taras Bobrovytsky > Assignee: Yongzhi Chen > Priority: Minor > Labels: ramp-up > Attachments: image-2020-02-07-11-31-38-074.png, > image-2020-02-07-11-31-43-641.png, image-2020-02-07-11-33-27-944.png > > > Impala cannot read this: > {code} > {"name": "tmp_1", > "type": "fixed", > "size": 8, > "logicalType": "decimal", > "precision": 10, > "scale": 5} > {code} > However, this can be read: > {code} > {"name": "tmp_1", > "type": "fixed", > "size": 5, > "logicalType": "decimal", > "precision": 10, > "scale": 5} > {code} > Size must be precisely set to this, or Impala is unable to read the decimal > column: > {code} > size = int(math.ceil((math.log(2, 10) + precision) / math.log(256, 10))) > {code} > There is nothing in the Parquet spec that says that Decimal columns must be > sized precisely. Arguably it's a bug in the writer if it's doing it, because > it's just wasting space. > https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#decimal -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org