ajantha-bhat commented on a change in pull request #3607: [CARBONDATA-3670]
Support compress offheap data in columnpage directly, avoding a copy of data
from offhead to heap before compressed.
URL: https://github.com/apache/carbondata/pull/3607#discussion_r376871815
##########
File path:
core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java
##########
@@ -747,6 +759,16 @@ public long getPageLengthInBytes() throws IOException {
*/
public byte[] compress(Compressor compressor) throws IOException {
DataType dataType = columnPageEncoderMeta.getStoreDataType();
+
+ // if the columnpage is isUnsafeEnabled and the Datatype is primitive.
+ // we try to compress the data in offheap directly, avoiding a copy from
offheap to heap
+ if (isUnsafeEnabled() && (dataType == DataTypes.BOOLEAN || dataType == BYTE
+ || dataType == SHORT || dataType == DataTypes.SHORT_INT || dataType ==
INT
+ || dataType == LONG || dataType == FLOAT || dataType == DOUBLE
+ || DataTypes.isDecimal(dataType))) {
Review comment:
is Decimal supported ?
below I see getByteBufferArrayPage is unsupported in DecimalColumnPage
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
