[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16147735#comment-16147735
]
ASF GitHub Bot commented on DRILL-5546:
---------------------------------------
Github user jinfengni commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r136149953
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java
---
@@ -130,562 +145,248 @@ public IterOutcome innerNext() {
}
@Override
- public WritableBatch getWritableBatch() {
- return WritableBatch.get(this);
+ public int getRecordCount() {
+ return recordCount;
}
- private void setValueCount(int count) {
- for (ValueVector v : allocationVectors) {
- ValueVector.Mutator m = v.getMutator();
- m.setValueCount(count);
- }
- }
- private boolean doAlloc() {
- for (ValueVector v : allocationVectors) {
- try {
- AllocationHelper.allocateNew(v, current.getRecordCount());
- } catch (OutOfMemoryException ex) {
- return false;
- }
+ @SuppressWarnings("resource")
+ private IterOutcome doWork(RecordBatch inputBatch, boolean newSchema)
throws ClassTransformationException, IOException, SchemaChangeException {
+ if (inputBatch.getSchema().getFieldCount() !=
container.getSchema().getFieldCount()) {
+ // wrong.
}
- return true;
- }
- @SuppressWarnings("resource")
- private IterOutcome doWork() throws ClassTransformationException,
IOException, SchemaChangeException {
- if (allocationVectors != null) {
- for (ValueVector v : allocationVectors) {
- v.clear();
- }
+ if (newSchema) {
+ createUnionAller(inputBatch);
}
- allocationVectors = Lists.newArrayList();
- transfers.clear();
+ container.zeroVectors();
- // If both sides of Union-All are empty
- if(unionAllInput.isBothSideEmpty()) {
- for(int i = 0; i < outputFields.size(); ++i) {
- final String colName = outputFields.get(i).getPath();
- final MajorType majorType = MajorType.newBuilder()
- .setMinorType(MinorType.INT)
- .setMode(DataMode.OPTIONAL)
- .build();
-
- MaterializedField outputField = MaterializedField.create(colName,
majorType);
- ValueVector vv = container.addOrGet(outputField, callBack);
- allocationVectors.add(vv);
- }
+ for (final ValueVector v : this.allocationVectors) {
+ AllocationHelper.allocateNew(v, inputBatch.getRecordCount());
+ }
- container.buildSchema(BatchSchema.SelectionVectorMode.NONE);
+ recordCount = unionall.unionRecords(0, inputBatch.getRecordCount(), 0);
+ for (final ValueVector v : allocationVectors) {
+ final ValueVector.Mutator m = v.getMutator();
+ m.setValueCount(recordCount);
+ }
--- End diff --
New util method in `VectorUtil`.
> Schema change problems caused by empty batch
> --------------------------------------------
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Jinfeng Ni
> Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
