[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035221#comment-16035221
]
Jinfeng Ni commented on DRILL-5546:
-----------------------------------
Thanks for reviewing the proposal.
Some clarifications.
1. The concept of *Empty Batch* (EB) defined in the proposal is exactly same
as what you described as *Null result set*. The batch does not contain data nor
schema.
2. In general, Drill operators are expected to handle *Empty result set*. As
you described, *Empty result set* could be produced by a Filter operator, or
any operator which does pruning, such as Join, Limit. I do not think there is a
fundamental problem for Drill operators to handle Empty result set.
3. AFAIK, some operator (UnionAll) may have bug to handle *Empty result set*.
The bug was introduced when trying to fix issues related to *Null result set*.
See the proposal for the related JIRA issues. The thought is with this
proposal on handling *Null result set*, we will be able to correct the bug in
operators such as UnionAll.
4. The semantics for different operators (Scan/Join/Union/Filter) handling
*Fast NONE* in the proposal seems to be same as what you describe in the
comment.
5. "So, the scanner should return an empty batch (with schema) if a reader
produces one (that is, skip null batches, return an empty batch.)" If *no*
reader produces one, the proposal is to return a *Fast NONE*, in stead of a
batch with injected nullable-int column.
6. Under *fast schema*, the first batch is same as a regular batch, if the
query only contains non-blocking operators. Only when query has blocking
operators such as HashJoin/HashAgg, the first batch going through after the
blocking operator contains schema only.
7. With the concept of *Fast NONE*, operator may not return any batch. If any
execution branch (minor fragment) returns a batch with schema, such batch will
still pop up along the execution pipeline, and there is no difference from the
perspective of *fast schema*.
8. The proposal discusses the special cases where all the minor fragments
return *fast NONE* (essentially entire dataset is empty).
> Schema change problems caused by empty batch
> --------------------------------------------
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Jinfeng Ni
> Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
