Repository: spark Updated Branches: refs/heads/branch-2.4 b6ba0dd47 -> 7f4fce426
[SPARK-25179][PYTHON][DOCS] Document BinaryType support in Arrow conversion ## What changes were proposed in this pull request? This PR targets to document binary type in "Apache Arrow in Spark". ## How was this patch tested? Manually built the documentation and checked. Closes #22871 from HyukjinKwon/SPARK-25179. Authored-by: hyukjinkwon <gurwls...@apache.org> Signed-off-by: gatorsmile <gatorsm...@gmail.com> (cherry picked from commit fbaf150507a289ec0ac02fdbf4009c42cd9bc164) Signed-off-by: gatorsmile <gatorsm...@gmail.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7f4fce42 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/7f4fce42 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/7f4fce42 Branch: refs/heads/branch-2.4 Commit: 7f4fce426025d54f41d8e87928582563a8ad689e Parents: b6ba0dd Author: hyukjinkwon <gurwls...@apache.org> Authored: Sun Oct 28 23:01:35 2018 -0700 Committer: gatorsmile <gatorsm...@gmail.com> Committed: Sun Oct 28 23:02:09 2018 -0700 ---------------------------------------------------------------------- docs/sql-pyspark-pandas-with-arrow.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/7f4fce42/docs/sql-pyspark-pandas-with-arrow.md ---------------------------------------------------------------------- diff --git a/docs/sql-pyspark-pandas-with-arrow.md b/docs/sql-pyspark-pandas-with-arrow.md index e8e9f55..d04b955 100644 --- a/docs/sql-pyspark-pandas-with-arrow.md +++ b/docs/sql-pyspark-pandas-with-arrow.md @@ -127,8 +127,9 @@ For detailed usage, please see [`pyspark.sql.functions.pandas_udf`](api/python/p ### Supported SQL Types -Currently, all Spark SQL data types are supported by Arrow-based conversion except `BinaryType`, `MapType`, -`ArrayType` of `TimestampType`, and nested `StructType`. +Currently, all Spark SQL data types are supported by Arrow-based conversion except `MapType`, +`ArrayType` of `TimestampType`, and nested `StructType`. `BinaryType` is supported only when +installed PyArrow is equal to or higher then 0.10.0. ### Setting Arrow Batch Size --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org