This is an automated email from the ASF dual-hosted git repository. srowen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 7e8aafd2c0f [SPARK-45334][SQL] Remove misleading comment in parquetSchemaConverter 7e8aafd2c0f is described below commit 7e8aafd2c0f1f6fcd03a69afe2b85fd3fda95d20 Author: lanmengran1 <lanmengr...@jd.com> AuthorDate: Tue Sep 26 21:01:02 2023 -0500 [SPARK-45334][SQL] Remove misleading comment in parquetSchemaConverter ### What changes were proposed in this pull request? Remove one line of comment, the detail info is described in JIRA https://issues.apache.org/jira/browse/SPARK-45334 ### Why are the changes needed? The comment is outdated and misleading. - the parquet-hive module has been removed from the parquet-mr project https://issues.apache.org/jira/browse/PARQUET-1676 - Hive always uses "array_element" as the name ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No need ### Was this patch authored or co-authored using generative AI tooling? No Closes #43119 from amoylan2/remove_misleading_comment_in_parquetSchemaConverter. Authored-by: lanmengran1 <lanmengr...@jd.com> Signed-off-by: Sean Owen <sro...@gmail.com> --- .../spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala | 1 - 1 file changed, 1 deletion(-) diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala index 9c9e7ce729c..eedd165278a 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala @@ -646,7 +646,6 @@ class SparkToParquetSchemaConverter( .buildGroup(repetition).as(LogicalTypeAnnotation.listType()) .addField(Types .buildGroup(REPEATED) - // "array" is the name chosen by parquet-hive (1.7.0 and prior version) .addField(convertField(StructField("array", elementType, nullable))) .named("bag")) .named(field.name) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org