Repository: spark Updated Branches: refs/heads/master 4e990d9dd -> fbaf15050
[SPARK-25179][PYTHON][DOCS] Document BinaryType support in Arrow conversion ## What changes were proposed in this pull request? This PR targets to document binary type in "Apache Arrow in Spark". ## How was this patch tested? Manually built the documentation and checked. Closes #22871 from HyukjinKwon/SPARK-25179. Authored-by: hyukjinkwon <gurwls...@apache.org> Signed-off-by: gatorsmile <gatorsm...@gmail.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fbaf1505 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/fbaf1505 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/fbaf1505 Branch: refs/heads/master Commit: fbaf150507a289ec0ac02fdbf4009c42cd9bc164 Parents: 4e990d9 Author: hyukjinkwon <gurwls...@apache.org> Authored: Sun Oct 28 23:01:35 2018 -0700 Committer: gatorsmile <gatorsm...@gmail.com> Committed: Sun Oct 28 23:01:35 2018 -0700 ---------------------------------------------------------------------- docs/sql-pyspark-pandas-with-arrow.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/fbaf1505/docs/sql-pyspark-pandas-with-arrow.md ---------------------------------------------------------------------- diff --git a/docs/sql-pyspark-pandas-with-arrow.md b/docs/sql-pyspark-pandas-with-arrow.md index e8e9f55..d04b955 100644 --- a/docs/sql-pyspark-pandas-with-arrow.md +++ b/docs/sql-pyspark-pandas-with-arrow.md @@ -127,8 +127,9 @@ For detailed usage, please see [`pyspark.sql.functions.pandas_udf`](api/python/p ### Supported SQL Types -Currently, all Spark SQL data types are supported by Arrow-based conversion except `BinaryType`, `MapType`, -`ArrayType` of `TimestampType`, and nested `StructType`. +Currently, all Spark SQL data types are supported by Arrow-based conversion except `MapType`, +`ArrayType` of `TimestampType`, and nested `StructType`. `BinaryType` is supported only when +installed PyArrow is equal to or higher then 0.10.0. ### Setting Arrow Batch Size --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
