[
https://issues.apache.org/jira/browse/DRILL-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17568391#comment-17568391
]
Paul Rogers commented on DRILL-8185:
------------------------------------
[~cgivre], thanks for the update! I believe union vectors do, in fact, work in
EVF V2. There are multiple unit tests for unions and lists (lists are like
arrays of unions.) There is even 2D and 3D list support. It turns out that what
was missing was the ability to have "dummy" versions so readers can write to
unprojected vectors and not know the difference. I wonder, what other problems
have you encountered? I also seem to recall getting unions and lists to work in
JSON, but I can't recall how complete the support is. The unit tests would give
the definitive answer.
Glad to hear the progress on unifying the JSON support. That was a desired goal
of moving to EVF. There are a number of older vector readers/writers as well
that, I believe, were only used by the JSON classes. Those older versions
didn't handle things like limiting vector sizes, or unprojected vectors, which
is one of the things that led to EVF in the first place.
The problem with union vectors is that they are not supported by most
operators. They are also not supported by most clients. Probably only the REST
client can handle unions. JDBC "fakes" it by converting values to strings. If
that was the end goal, might as well do that at read time and save the hassle
of unions.
[~vitalii] may find the need to test, and perhaps fix, other operators. For
example, the sort operator can't sort on a union: it doesn't know how to
compare a string and a float, say. The aggregate operators may not know how to
combine values of different types, nor can the group operator create meaningful
groups.
Over the years, we fixed MANY problems in unions and lists. Every time I tried
to get them to work with EVF, I found new holes in the vector implementations.
Many of those issues are now fixed, which is why the unit tests for EVF for
unions and lists pass. (The tests use the term "variant", so look for that if
looking for the tests.)
> EVF 2 doen't handle map arrays or nested maps
> ---------------------------------------------
>
> Key: DRILL-8185
> URL: https://issues.apache.org/jira/browse/DRILL-8185
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.20.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Major
> Fix For: 2.0.0
>
>
> When converting Avro, Luoc found two bugs in how EVF 2 (the projection
> mechanism) handles map array and nested maps
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
