[
https://issues.apache.org/jira/browse/DRILL-6262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405603#comment-16405603
]
ASF GitHub Bot commented on DRILL-6262:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1175#discussion_r175619764
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java
---
@@ -321,10 +321,8 @@ public ColumnSize(ValueVector v, String prefix) {
// Calculate pure data size.
if (isVariableWidth) {
- UInt4Vector offsetVector = ((RepeatedValueVector)
v).getOffsetVector();
- int innerValueCount =
offsetVector.getAccessor().get(valueCount);
VariableWidthVector dataVector = ((VariableWidthVector)
((RepeatedValueVector) v).getDataVector());
- totalDataSize =
dataVector.getOffsetVector().getAccessor().get(innerValueCount);
+ totalDataSize = dataVector.getCurrentSizeInBytes();
--- End diff --
Good improvement. The original code exposes far too much of the
implementation.
After all these changes, does the "dataSize" include both the offset vector
and bytes? It should, else calls will be wrong. There are supposed to be three
sizes:
* Payload size: actual data bytes.
* Data size: data + offsets + bits
* Overall size: full length of all vectors.
Payload size is what the user sees. Data size is how we calculate row width
(since the rows must contain the overhead bytes). Vector length, here, only
helps compute density, but is generated elsewhere. The point is, keep all three
in mind, but keep the code separate. Otherwise, it is *very* easy to get
confused and have the calculations blow up...
> IndexOutOfBoundException in RecordBatchSize for empty variableWidthVector
> -------------------------------------------------------------------------
>
> Key: DRILL-6262
> URL: https://issues.apache.org/jira/browse/DRILL-6262
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Reporter: Sorabh Hamirwasia
> Assignee: Sorabh Hamirwasia
> Priority: Major
> Labels: ready-to-commit
> Fix For: 1.14.0
>
>
> ColumnSize inside RecordBatchSizer while computing the totalDataSize for
> VariableWidthVector throws IndexOutOfBoundException when the underlying
> vector is empty without any allocated memory.
> This happens because the way totalDataSize is computed is using the
> offsetVector value at an index n where n is total number of records in the
> vector. When vector is empty then n=0 and offsetVector drillbuf is empty as
> well. So while retrieving value at index 0 from offsetVector exception is
> thrown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
