Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2417#discussion_r200842962
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/ColumnPageWrapper.java
---
@@ -53,11 +60,75 @@ public int fillVector(int[] filteredRowId,
ColumnVectorInfo[] vectorInfo, int ch
throw new UnsupportedOperationException("internal error");
}
- @Override
- public byte[] getChunkData(int rowId) {
- return columnPage.getBytes(rowId);
+ @Override public byte[] getChunkData(int rowId) {
+ ColumnType columnType = columnPage.getColumnSpec().getColumnType();
+ DataType srcDataType = columnPage.getColumnSpec().getSchemaDataType();
+ DataType targetDataType = columnPage.getDataType();
+ if (columnPage.getNullBits().get(rowId)) {
+ // if this row is null, return default null represent in byte array
+ return CarbonCommonConstants.MEMBER_DEFAULT_VAL_ARRAY;
+ }
+ if ((columnType == ColumnType.COMPLEX_PRIMITIVE) &&
this.isAdaptiveComplexPrimitive()) {
+ if (srcDataType == DataTypes.DOUBLE || srcDataType ==
DataTypes.FLOAT) {
+ double doubleData = columnPage.getDouble(rowId);
+ if (srcDataType == DataTypes.FLOAT) {
+ float out = (float) doubleData;
+ return ByteUtil.toBytes(out);
+ } else {
+ return ByteUtil.toBytes(doubleData);
+ }
+ } else if (DataTypes.isDecimal(srcDataType)) {
+ throw new RuntimeException("unsupported type: " + srcDataType);
+ } else if ((srcDataType == DataTypes.BYTE) ||
+ (srcDataType == DataTypes.BOOLEAN) ||
+ (srcDataType == DataTypes.SHORT) ||
+ (srcDataType == DataTypes.SHORT_INT) ||
+ (srcDataType == DataTypes.INT) ||
+ (srcDataType == DataTypes.LONG) ||
+ (srcDataType == DataTypes.TIMESTAMP)) {
+ long longData = columnPage.getLong(rowId);
--- End diff --
Should we read the bytes from column page based type ? Otherwise for small
types like byte,short, also reading long would consume 8 bytes from the page
which leads wrong data?
---
