[
https://issues.apache.org/jira/browse/DRILL-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543439#comment-16543439
]
ASF GitHub Bot commented on DRILL-5495:
---------------------------------------
vdiravka commented on issue #1382: DRILL-5495: convert_from function on top of
int96 data results in Arr…
URL: https://github.com/apache/drill/pull/1382#issuecomment-404892801
@arina-ielchiieva Yes, it is mechanical issue.
`recordsToReadInThisPass` is `numNonNullValues` (see
`NullableColumnReader#processPagesBulk():284`)
But `recordsReadInThisIteration` is `numNullValues` + `numNonNullValues`.
In `NullableColumnReader#readField()` the only `numNonNullValues` should be
used.
It is hard to reproduce the issue with one small file. But since it was the
mechanical error, I think it is fine do not add unit test for this issue. I
have updated the PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> convert_from function on top of int96 data results in
> ArrayIndexOutOfBoundsException
> ------------------------------------------------------------------------------------
>
> Key: DRILL-5495
> URL: https://issues.apache.org/jira/browse/DRILL-5495
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.10.0
> Reporter: Rahul Challapalli
> Assignee: Vitalii Diravka
> Priority: Major
> Fix For: 1.14.0
>
> Attachments: 26edf56f-6bc6-1e1f-5aa4-d98aec858a4a.sys.drill,
> d4.tar.gz, drillbit.log
>
>
> git.commit.id.abbrev=1e0a14c
> The data set used is generated from spark and contains a timestamp stored as
> int96
> {code}
> [root@qa-node190 framework]# /home/parquet-tools-1.5.1-SNAPSHOT/parquet-meta
> /home/framework/framework/resources/Datasources/parquet_date/spark_generated/d4/part-r-00000-08c5c621-62ea-4fee-b690-11576eddc39c.snappy.parquet
>
> creator: parquet-mr (build 32c46643845ea8a705c35d4ec8fc654cc8ff816d)
> extra: org.apache.spark.sql.parquet.row.metadata =
> {"type":"struct","fields":[{"name":"a","type":"integer","nullable":true,"metadata":{}},{"name":"b","type":"strin
> [more]...
> file schema: spark_schema
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> a: OPTIONAL INT32 R:0 D:1
> b: OPTIONAL BINARY O:UTF8 R:0 D:1
> c: OPTIONAL INT32 O:DATE R:0 D:1
> d: OPTIONAL INT96 R:0 D:1
> row group 1: RC:10000 TS:8661
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> a: INT32 SNAPPY DO:0 FPO:4 SZ:2367/2571/1.09 VC:10000
> ENC:RLE,PLAIN,BIT_PACKED
> b: BINARY SNAPPY DO:0 FPO:2371 SZ:2329/2843/1.22 VC:10000
> ENC:RLE,PLAIN_DICTIONARY,BIT_PACKED
> c: INT32 SNAPPY DO:0 FPO:4700 SZ:1374/1507/1.10 VC:10000
> ENC:RLE,PLAIN,BIT_PACKED
> d: INT96 SNAPPY DO:0 FPO:6074 SZ:1597/1740/1.09 VC:10000
> ENC:RLE,PLAIN_DICTIONARY,BIT_PACKED
> {code}
> The below query fails with an ArrayIndexOutOfBoundsException
> {code}
> select convert_from(d, 'TIMESTAMP_IMPALA') from
> dfs.`/drill/testdata/resource-manager/d4`;
> Fails with below error after displaying a bunch of records
> Error: SYSTEM ERROR: ArrayIndexOutOfBoundsException: 0
> Fragment 1:0
> [Error Id: f963f6c0-3306-49a6-9d98-a193c5e7cfee on qa-node190.qa.lab:31010]
> (state=,code=0)
> {code}
> Attached the logs, profiles and data files
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
