[CARBONDATA-1490] Fixed memory allocation for carbon row While creating carbon row the space for measures is assigned based on outputLength and not on measureCount which will tend to allocate more space than is actually needed.
outputLength = measureCount + ((noDictCount + complexCount) > 0 ? 1 : 0) + 1. When the table has 1 complex column then measureCount=0, noDictCount=0, complexCount=1 therefore, outputLength = 2. Even if there are no measure columns the measure object is created with 2 null values. This closes #1373 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/d4eabbe5 Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/d4eabbe5 Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/d4eabbe5 Branch: refs/heads/branch-1.2 Commit: d4eabbe56aa6825d84933ab5a0fc78f26762cea9 Parents: e9c24c5 Author: kunal642 <kunalkapoor...@gmail.com> Authored: Wed Sep 20 11:02:00 2017 +0530 Committer: manishgupta88 <tomanishgupt...@gmail.com> Committed: Wed Sep 20 14:41:38 2017 +0530 ---------------------------------------------------------------------- .../newflow/steps/CarbonRowDataWriterProcessorStepImpl.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/carbondata/blob/d4eabbe5/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java ---------------------------------------------------------------------- diff --git a/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java b/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java index 765e0ed..643d2ba 100644 --- a/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java +++ b/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java @@ -254,7 +254,7 @@ public class CarbonRowDataWriterProcessorStepImpl extends AbstractDataLoadProces nonDicArray[nonDicIndex++] = (byte[]) row.getObject(dimCount); } - Object[] measures = new Object[outputLength]; + Object[] measures = new Object[measureCount]; for (int i = 0; i < this.measureCount; i++) { measures[i] = row.getObject(i + this.dimensionWithComplexCount); }
