[
https://issues.apache.org/jira/browse/DRILL-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417342#comment-16417342
]
ASF GitHub Bot commented on DRILL-6230:
---------------------------------------
Github user ppadma commented on a diff in the pull request:
https://github.com/apache/drill/pull/1161#discussion_r177175974
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/test/rowSet/HyperRowSetImpl.java
---
@@ -45,8 +50,67 @@ public RowSetReader buildReader(HyperRowSet rowSet,
SelectionVector4 sv4) {
TupleMetadata schema = rowSet.schema();
HyperRowIndex rowIndex = new HyperRowIndex(sv4);
return new RowSetReaderImpl(schema, rowIndex,
- buildContainerChildren(rowSet.container(),
- new MetadataRetrieval(schema)));
+ buildContainerChildren(rowSet.container(), schema));
+ }
+ }
+
+ public static class HyperRowSetBuilderImpl implements HyperRowSetBuilder
{
+
+ private final BufferAllocator allocator;
+ private final List<VectorContainer> batches = new ArrayList<>();
+ private int totalRowCount;
+
+ public HyperRowSetBuilderImpl(BufferAllocator allocator) {
+ this.allocator = allocator;
+ }
+
+ @Override
+ public void addBatch(SingleRowSet rowSet) {
+ if (rowSet.rowCount() == 0) {
+ return;
+ }
+ if (rowSet.indirectionType() != SelectionVectorMode.NONE) {
+ throw new IllegalArgumentException("Batches must not have a
selection vector.");
+ }
+ batches.add(rowSet.container());
+ totalRowCount += rowSet.rowCount();
+ }
+
+ @Override
+ public void addBatch(VectorContainer container) {
+ if (container.getRecordCount() == 0) {
+ return;
+ }
+ if (container.getSchema().getSelectionVectorMode() !=
SelectionVectorMode.NONE) {
+ throw new IllegalArgumentException("Batches must not have a
selection vector.");
+ }
+ batches.add(container);
+ totalRowCount += container.getRecordCount();
+ }
+
+ @SuppressWarnings("resource")
+ @Override
+ public HyperRowSet build() throws SchemaChangeException {
+ SelectionVector4 sv4 = new SelectionVector4(allocator,
totalRowCount);
+ ExpandableHyperContainer hyperContainer = new
ExpandableHyperContainer();
+ for (VectorContainer container : batches) {
+ hyperContainer.addBatch(container);
+ }
+
+ // TODO: This has a bug. If the hyperset has two batches with unions,
+ // and the first union contains only VARCHAR, while the second
contains
--- End diff --
is there a JIRA for this bug ?
> Extend row set readers to handle hyper vectors
> ----------------------------------------------
>
> Key: DRILL-6230
> URL: https://issues.apache.org/jira/browse/DRILL-6230
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Major
> Fix For: 1.14.0
>
>
> The current row set readers have incomplete support for hyper-vectors. To add
> full support, we need an interface that supports either single batches or
> hyper batches. Accessing vectors in hyper batches differs depending on
> whether the vector is at the top level or is nested. See [this
> post|https://github.com/paul-rogers/drill/wiki/BH-Column-Readers] for
> details. Also includes a simpler reader template: replaces the original three
> classes with one, in parallel with the writers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
