[
https://issues.apache.org/jira/browse/DRILL-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-1962:
---------------------------------
Fix Version/s: 0.9.0
> accessing nested array from multiple files causing IndexOutOfBoundException
> ---------------------------------------------------------------------------
>
> Key: DRILL-1962
> URL: https://issues.apache.org/jira/browse/DRILL-1962
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
> Affects Versions: 0.8.0
> Reporter: Chun Chang
> Assignee: Parth Chandra
> Priority: Critical
> Fix For: 0.9.0
>
>
> #Wed Jan 07 18:54:07 EST 2015
> git.commit.id.abbrev=35a350f
> If the dataset contains nested array of array, and the data is contained in
> more than one file, accessing the second nested array cause
> IndexOutOfBoundsException. For example, with the following dataset:
> {code}
> {
> "id": 2,
> "oooa": {
> "oa": {
> "oab": {
> "oabc": [
> {
> "rowId": 2
> },
> {
> "rowValue1": [{"rv1":1, "rv2":2}, {"rva1":3,
> "rva2":4}],
> "rowValue2": [{"rw1":1, "rw2":2}, {"rwa1":3,
> "rwa2":4}]
> }
> ]
> }
> }
> }
> }
> {code}
> If you put it in two separate files in the same directory, query using wild
> card to accessing the two files at the second array level will cause the
> exception.
> {code}
> 0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select
> t.oooa.oa.oab.oabc[1].rowValue1 from `jira2file/jira*.json` t;
> +------------+
> | EXPR$0 |
> +------------+
> | [{"rv1":1,"rv2":2},{"rva1":3,"rva2":4}] |
> Query failed: Query failed: Failure while running fragment., index: -4,
> length: 4 (expected: range(0, 16384)) [ 78235243-4f01-4ee3-9675-fc18bd1e66e3
> on qa-node120.qa.lab:31010 ]
> [ 78235243-4f01-4ee3-9675-fc18bd1e66e3 on qa-node120.qa.lab:31010 ]
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing
> query.
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> 0: jdbc:drill:schema=dfs.drillTestDirComplexJ>
> {code}
> Same query on single file works.
> {code}
> 0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select
> t.oooa.oa.oab.oabc[1].rowValue1 from `jira2file/jira1.json` t;
> +------------+
> | EXPR$0 |
> +------------+
> | [{"rv1":1,"rv2":2},{"rva1":3,"rva2":4}] |
> +------------+
> {code}
> stack trace:
> {code}
> 2015-01-08 14:00:16,127 [2b51020e-daab-a903-ef7c-ef6f9bd606c7:frag:0:0] WARN
> o.a.d.e.w.fragment.FragmentExecutor - Error while initializing or executing
> fragment
> java.lang.IndexOutOfBoundsException: index: -4, length: 4 (expected: range(0,
> 16384))
> at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
> at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
> at
> org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:297)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:542)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setChildrenPosition(RepeatedMapReaderImpl.java:45)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:96)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.test.generated.ProjectorGen4732.doEval(ProjectorTemplate.java:30)
> ~[na:na]
> at
> org.apache.drill.exec.test.generated.ProjectorGen4732.projectRecords(ProjectorTemplate.java:64)
> ~[na:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:172)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:97)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:114)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:254)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> {code}
> physical plan:
> {code}
> 0: jdbc:drill:schema=dfs.drillTestDirComplexJ> explain plan for select
> t.oooa.oa.oab.oabc[1].rowValue1 from `jira2file/jira*.json` t;
> +------------+------------+
> | text | json |
> +------------+------------+
> | 00-00 Screen
> 00-01 Project(EXPR$0=[ITEM(ITEM(ITEM(ITEM(ITEM($0, 'oa'), 'oab'),
> 'oabc'), 1), 'rowValue1')])
> 00-02 Scan(groupscan=[EasyGroupScan
> [selectionRoot=/drill/testdata/complex_type/json/jira2file, numFiles=2,
> columns=[`oooa`.`oa`.`oab`.`oabc`[1].`rowValue1`],
> files=[maprfs:/drill/testdata/complex_type/json/jira2file/jira1.json,
> maprfs:/drill/testdata/complex_type/json/jira2file/jira2.json]]])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
