[ https://issues.apache.org/jira/browse/SPARK-12785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101520#comment-15101520 ]
Tim Preece commented on SPARK-12785: ------------------------------------ I notice there is an explicit check for endianess in OffHeapColumnVector. Is the a technical reason why this feature is only being developed for Little Endian platforms ? I think the endianess should never matter ( providing Spark is run on an homogeneous endianess cluster ). > Implement columnar in memory representation > ------------------------------------------- > > Key: SPARK-12785 > URL: https://issues.apache.org/jira/browse/SPARK-12785 > Project: Spark > Issue Type: New Feature > Components: SQL > Reporter: Nong Li > Assignee: Nong Li > Fix For: 2.0.0 > > > Tungsten can benefit from having a columnar in memory representation which > can provide a few benefits: > - Enables vectorized execution > - Improves memory efficiency (memory is more tightly packed) > - Enables cheap serialization/zero-copy transfer with third party components > (e.g. numpy) > -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org