[
https://issues.apache.org/jira/browse/DRILL-8375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17803315#comment-17803315
]
ASF GitHub Bot commented on DRILL-8375:
---------------------------------------
paul-rogers opened a new pull request, #2867:
URL: https://github.com/apache/drill/pull/2867
# Support for non-projected complex vectors
## Description
The EVF mechanism provides scan-time projection for many vector types. The
reader code is simple: it deserializes all columns for formats such as JSON,
CSV, etc., and writes them to the `ColumnWriter` objects. Internally, EVF
simply ignores the data for unprojected columns. This solution simplifies the
readers: it is not necessary for each reader to include the complex code to
handle projection. This solution is also performant: projection is done at scan
time rather than the other approach, which is to read all data into vectors,
then allow a PROJECT operator to drop the unprojected columns.
Present EVF projection support handles most scalar and "well-structured"
columns (repeated types AKA arrays, maps, etc.) However it does not handle the
more esoteric types UNION, LIST (AKA repeated UNION), REPEATED LIST (AKA
repeated, repeated UNION). This PR provides more support, though holes remain.
## Documentation
This is an internal feature: no user-visible documentation is required.
## Testing
Extended existing EVF-related unit tests.
> Incomplete support for non-projected complex vectors
> ----------------------------------------------------
>
> Key: DRILL-8375
> URL: https://issues.apache.org/jira/browse/DRILL-8375
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Major
>
> The `ResultSetLoader` implementation supports all of Drill's vector types.
> However, DRILL-8188 discovered holes in support for non-projected vectors.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
