This is an automated email from the ASF dual-hosted git repository.
lixiao pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 9d307dd [MINOR][DOC] Avro data source documentation change
9d307dd is described below
commit 9d307dd53857e3eac9509992505be5ca37f2de7a
Author: Jules Damji <dmat...@comcast.net>
AuthorDate: Tue Jun 4 16:17:53 2019 -0700
[MINOR][DOC] Avro data source documentation change
This is a minor documentation change whereby the
https://spark.apache.org/docs/latest/sql-data-sources-avro.html mentions "The
date type and naming of record fields should match the input Avro data or
Catalyst data,"
The term Catalyst data is confusing. It should instead say, Spark's
internal data type such as String
Type or IntegerType.
(Please explain how this patch was tested. E.g. unit tests, integration
tests, manual tests)
There are no code changes; only doc changes.
Please review https://spark.apache.org/contributing.html before opening a
pull request.
Closes #24787 from dmatrix/br-orc-ds.doc.changes.
Authored-by: Jules Damji <dmat...@comcast.net>
Signed-off-by: gatorsmile <gatorsm...@gmail.com>
(cherry picked from commit b71abd654de2886ff2b44cada81ea909a0712f7c)
Signed-off-by: gatorsmile <gatorsm...@gmail.com>
---
docs/sql-data-sources-avro.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/sql-data-sources-avro.md b/docs/sql-data-sources-avro.md
index d3b81f0..285b3aa 100644
--- a/docs/sql-data-sources-avro.md
+++ b/docs/sql-data-sources-avro.md
@@ -148,8 +148,8 @@ Data source options of Avro can be set using the `.option`
method on `DataFrameR
<tr>
<td><code>avroSchema</code></td>
<td>None</td>
- <td>Optional Avro schema provided by an user in JSON format. The date type
and naming of record fields
- should match the input Avro data or Catalyst data, otherwise the
read/write action will fail.</td>
+ <td>Optional Avro schema provided by a user in JSON format. The data type
and naming of record fields
+ should match the Avro data type when reading from Avro or match the
Spark's internal data type (e.g., StringType, IntegerType) when writing to Avro
files; otherwise, the read/write action will fail.</td>
<td>read and write</td>
</tr>
<tr>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
