Hi All, Happy New Year!
I dusted off the work to add non-projection support in EVF for the UNION and LIST types. I believe that only REPEATED LIST is missing. "Non-projection support" just means that you can read a JSON file that requires a UNION or LIST vector, and tell EVF to NOT actually project that vector. That is, if the union column is u and the list is l, and you have other columns a, b, c, you can write "SELECT a, b, c FROM 'myFile.json'". JSON wil read all columns, but EVF won't actually create vectors for the columns u and l that you didn't want. If you write a format plugin, your code can write to the "dummy" UNION column writer, blissfully ignorant of the fact that there is no actual vector underneath: the data just goes into the bit bucket. The work mostly required fiddling with existing mechanisms. A new test file demonstrates that the changes work. All the existing tests suggest that the work didn't break anything. Reviews greatly appreciated. https://github.com/apache/drill/pull/2867 It would be great to get this into the next release, but doing so isn't critical, unless Luoc still needs this work. Thanks, - Paul
