[
https://issues.apache.org/jira/browse/DRILL-1324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14194275#comment-14194275
]
Mehant Baid commented on DRILL-1324:
------------------------------------
Updated patch with your review comments, answers inline:
1. If SchemaChangeCallBack is applicable to Map/RepeatedList/RepeatedMap only,
is it a good idea to define "callback" in AbstractContainerVector. ( Seems the
method of addOrGet() in map/RepeatedMap are identical?)
- Done
2. RepeatedList would pass "callback" to its children, while Map/RepeatedMap
would not. Is this the wanted behavior? That is, we do not want to detect a new
filed added in a map nested in another map / repeated map. Does it make sense
to implement two choices : one only detects the top level schema change, the
other one detects schema change recursively ? The caller who creates the vector
should tell the vector which behavior it wants.
- Good catch, we want to detect schema changes at all levels. Added callback
parameter when Map and Repeated map vectors are creating child elements.
3. The patch changes ProjectRecordBatch, and does not touch the other operator.
Does it mean only Project care schema change within a map, or other operators
may also care?
- Scan and Project are the only two operators that are allowed to return
OK_NEW_SCHEMA without its upstream return OK_NEW_SCHEMA. So currently Scan and
Project are the ones using this callback mechanism.
4. In ProjectRecordBatch, "callback" is passed down for the "evaluation" case.
Do we need consider complex writer case?
- Added callback to the complex writer case as well and a few other cases when
we might create a new vector in Project.
> Detect schema changes when complex vector changes internally
> -------------------------------------------------------------
>
> Key: DRILL-1324
> URL: https://issues.apache.org/jira/browse/DRILL-1324
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Mehant Baid
> Assignee: Mehant Baid
> Fix For: 0.6.0
>
> Attachments: DRILL-1324.patch
>
>
> Currently we indicate a schema change only if the top level vector in the
> container changes. We need to detect changes when we add a vector to the top
> level Map, Repeated Map, Repeated List in the container.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
