[
https://issues.apache.org/jira/browse/DRILL-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543499#comment-16543499
]
ASF GitHub Bot commented on DRILL-5495:
---------------------------------------
arina-ielchiieva closed pull request #1382: DRILL-5495: convert_from function
on top of int96 data results in Arr…
URL: https://github.com/apache/drill/pull/1382
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java
index 6a09bd64259..89aa8083fb2 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java
@@ -81,17 +81,16 @@ protected void readField(long recordsToReadInThisPass) {
if (usingDictionary) {
NullableVarBinaryVector.Mutator mutator = valueVec.getMutator();
Binary currDictValToWrite;
- for (int i = 0; i < recordsReadInThisIteration; i++){
+ for (int i = 0; i < recordsToReadInThisPass; i++) {
currDictValToWrite = pageReader.dictionaryValueReader.readBytes();
ByteBuffer buf = currDictValToWrite.toByteBuffer();
- mutator.setSafe(valuesReadInCurrentPass + i, buf, buf.position(),
- currDictValToWrite.length());
+ mutator.setSafe(valuesReadInCurrentPass + i, buf, buf.position(),
currDictValToWrite.length());
}
// Set the write Index. The next page that gets read might be a page
that does not use dictionary encoding
// and we will go into the else condition below. The readField method
of the parent class requires the
// writer index to be set correctly.
int writerIndex = castedBaseVector.getBuffer().writerIndex();
- castedBaseVector.getBuffer().setIndex(0, writerIndex +
(int)readLength);
+ castedBaseVector.getBuffer().setIndex(0, writerIndex + (int)
readLength);
} else {
super.readField(recordsToReadInThisPass);
// TODO - replace this with fixed binary type in drill
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> convert_from function on top of int96 data results in
> ArrayIndexOutOfBoundsException
> ------------------------------------------------------------------------------------
>
> Key: DRILL-5495
> URL: https://issues.apache.org/jira/browse/DRILL-5495
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.10.0
> Reporter: Rahul Challapalli
> Assignee: Vitalii Diravka
> Priority: Major
> Labels: ready-to-commit
> Fix For: 1.14.0
>
> Attachments: 26edf56f-6bc6-1e1f-5aa4-d98aec858a4a.sys.drill,
> d4.tar.gz, drillbit.log
>
>
> git.commit.id.abbrev=1e0a14c
> The data set used is generated from spark and contains a timestamp stored as
> int96
> {code}
> [root@qa-node190 framework]# /home/parquet-tools-1.5.1-SNAPSHOT/parquet-meta
> /home/framework/framework/resources/Datasources/parquet_date/spark_generated/d4/part-r-00000-08c5c621-62ea-4fee-b690-11576eddc39c.snappy.parquet
>
> creator: parquet-mr (build 32c46643845ea8a705c35d4ec8fc654cc8ff816d)
> extra: org.apache.spark.sql.parquet.row.metadata =
> {"type":"struct","fields":[{"name":"a","type":"integer","nullable":true,"metadata":{}},{"name":"b","type":"strin
> [more]...
> file schema: spark_schema
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> a: OPTIONAL INT32 R:0 D:1
> b: OPTIONAL BINARY O:UTF8 R:0 D:1
> c: OPTIONAL INT32 O:DATE R:0 D:1
> d: OPTIONAL INT96 R:0 D:1
> row group 1: RC:10000 TS:8661
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> a: INT32 SNAPPY DO:0 FPO:4 SZ:2367/2571/1.09 VC:10000
> ENC:RLE,PLAIN,BIT_PACKED
> b: BINARY SNAPPY DO:0 FPO:2371 SZ:2329/2843/1.22 VC:10000
> ENC:RLE,PLAIN_DICTIONARY,BIT_PACKED
> c: INT32 SNAPPY DO:0 FPO:4700 SZ:1374/1507/1.10 VC:10000
> ENC:RLE,PLAIN,BIT_PACKED
> d: INT96 SNAPPY DO:0 FPO:6074 SZ:1597/1740/1.09 VC:10000
> ENC:RLE,PLAIN_DICTIONARY,BIT_PACKED
> {code}
> The below query fails with an ArrayIndexOutOfBoundsException
> {code}
> select convert_from(d, 'TIMESTAMP_IMPALA') from
> dfs.`/drill/testdata/resource-manager/d4`;
> Fails with below error after displaying a bunch of records
> Error: SYSTEM ERROR: ArrayIndexOutOfBoundsException: 0
> Fragment 1:0
> [Error Id: f963f6c0-3306-49a6-9d98-a193c5e7cfee on qa-node190.qa.lab:31010]
> (state=,code=0)
> {code}
> Attached the logs, profiles and data files
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
