[
https://issues.apache.org/jira/browse/DRILL-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545598#comment-16545598
]
ASF GitHub Bot commented on DRILL-6475:
---------------------------------------
amansinha100 commented on a change in pull request #1381: DRILL-6475: Unnest:
Null fieldId Pointer.
URL: https://github.com/apache/drill/pull/1381#discussion_r202781143
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/JoinPrelRenameVisitor.java
##########
@@ -19,36 +19,70 @@
import java.util.ArrayList;
import java.util.List;
-
+import java.util.Map;
+import java.util.HashMap;
+
+import com.google.common.base.Preconditions;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeField;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexNode;
import org.apache.drill.exec.planner.physical.JoinPrel;
import org.apache.drill.exec.planner.physical.LateralJoinPrel;
import org.apache.drill.exec.planner.physical.Prel;
import org.apache.calcite.rel.RelNode;
import com.google.common.collect.Lists;
+import org.apache.drill.exec.planner.physical.UnnestPrel;
public class JoinPrelRenameVisitor extends BasePrelVisitor<Prel, Void,
RuntimeException>{
+ private final Map<String, Prel> sourceOperatorRegistry = new HashMap();
+
private static JoinPrelRenameVisitor INSTANCE = new JoinPrelRenameVisitor();
public static Prel insertRenameProject(Prel prel){
return prel.accept(INSTANCE, null);
}
+ private void register(Prel toRegister) {
+ this.sourceOperatorRegistry.put(toRegister.getClass().getSimpleName(),
toRegister);
Review comment:
One option is to have a specialized PrelVisitor that does the traversal
(similar to what we do for a few other things in physical planning) and for the
right child keeps track of the nearest ancestor Lateral and if it encounters an
Unnest, it ties up the two together. In fact the Materalizer already does this
for the POPs but for your use case this is needed earlier in the planning
phase.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Unnest: Null fieldId Pointer
> -----------------------------
>
> Key: DRILL-6475
> URL: https://issues.apache.org/jira/browse/DRILL-6475
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
> Reporter: Boaz Ben-Zvi
> Assignee: Hanumath Rao Maduri
> Priority: Major
> Fix For: 1.14.0
>
>
> Executing the following (in TestE2EUnnestAndLateral.java) causes an NPE as
> `fieldId` is null in `schemaChanged()`:
> {code}
> @Test
> public void testMultipleBatchesLateral_twoUnnests() throws Exception {
> String sql = "SELECT t5.l_quantity FROM dfs.`lateraljoin/multipleFiles/` t,
> LATERAL " +
> "(SELECT t2.ordrs FROM UNNEST(t.c_orders) t2(ordrs)) t3(ordrs), LATERAL " +
> "(SELECT t4.l_quantity FROM UNNEST(t3.ordrs) t4(l_quantity)) t5";
> test(sql);
> }
> {code}
>
> And the error is:
> {code}
> Error: SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: 25f42765-8f68-418e-840a-ffe65788e1e2 on 10.254.130.25:31020]
> (java.lang.NullPointerException) null
>
> org.apache.drill.exec.physical.impl.unnest.UnnestRecordBatch.schemaChanged():381
> org.apache.drill.exec.physical.impl.unnest.UnnestRecordBatch.innerNext():199
> org.apache.drill.exec.record.AbstractRecordBatch.next():172
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():229
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
>
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.prefetchFirstBatchFromBothSides():241
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.buildSchema():264
> org.apache.drill.exec.record.AbstractRecordBatch.next():152
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():229
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
> org.apache.drill.exec.record.AbstractRecordBatch.next():109
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():137
> org.apache.drill.exec.record.AbstractRecordBatch.next():172
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():229
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
> org.apache.drill.exec.record.AbstractRecordBatch.next():109
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():137
> org.apache.drill.exec.record.AbstractRecordBatch.next():172
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():229
> org.apache.drill.exec.physical.impl.BaseRootExec.next():103
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():83
> org.apache.drill.exec.physical.impl.BaseRootExec.next():93
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():292
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():279
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1657
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():279
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
