This is an automated email from the ASF dual-hosted git repository. lixiao pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push: new 9d307dd [MINOR][DOC] Avro data source documentation change 9d307dd is described below commit 9d307dd53857e3eac9509992505be5ca37f2de7a Author: Jules Damji <dmat...@comcast.net> AuthorDate: Tue Jun 4 16:17:53 2019 -0700 [MINOR][DOC] Avro data source documentation change This is a minor documentation change whereby the https://spark.apache.org/docs/latest/sql-data-sources-avro.html mentions "The date type and naming of record fields should match the input Avro data or Catalyst data," The term Catalyst data is confusing. It should instead say, Spark's internal data type such as String Type or IntegerType. (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) There are no code changes; only doc changes. Please review https://spark.apache.org/contributing.html before opening a pull request. Closes #24787 from dmatrix/br-orc-ds.doc.changes. Authored-by: Jules Damji <dmat...@comcast.net> Signed-off-by: gatorsmile <gatorsm...@gmail.com> (cherry picked from commit b71abd654de2886ff2b44cada81ea909a0712f7c) Signed-off-by: gatorsmile <gatorsm...@gmail.com> --- docs/sql-data-sources-avro.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sql-data-sources-avro.md b/docs/sql-data-sources-avro.md index d3b81f0..285b3aa 100644 --- a/docs/sql-data-sources-avro.md +++ b/docs/sql-data-sources-avro.md @@ -148,8 +148,8 @@ Data source options of Avro can be set using the `.option` method on `DataFrameR <tr> <td><code>avroSchema</code></td> <td>None</td> - <td>Optional Avro schema provided by an user in JSON format. The date type and naming of record fields - should match the input Avro data or Catalyst data, otherwise the read/write action will fail.</td> + <td>Optional Avro schema provided by a user in JSON format. The data type and naming of record fields + should match the Avro data type when reading from Avro or match the Spark's internal data type (e.g., StringType, IntegerType) when writing to Avro files; otherwise, the read/write action will fail.</td> <td>read and write</td> </tr> <tr> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org