Takuya Ueshin created SPARK-21440: ------------------------------------- Summary: Refactor ArrowConverters and add DecimalType, ArrayType and StructType support. Key: SPARK-21440 URL: https://issues.apache.org/jira/browse/SPARK-21440 Project: Spark Issue Type: Sub-task Components: PySpark, SQL Affects Versions: 2.3.0 Reporter: Takuya Ueshin
This is a refactoring of {{ArrowConverters}} and related classes. # Introduce {{ArrowColumnVector}} to read Arrow data in Java. # Refactor {{ColumnWriter}} as {{ArrowWriter}}. # Add {{DecimalType}}, {{ArrayType}} and {{StructType}} support. # Refactor {{ArrowConverters}} to skip intermediate {{ArrowRecordBatch}} creation. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org