Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19773#discussion_r218324420
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -350,16 +366,11 @@ case class AlterTableChangeColumnCommand(
s"${schema.fieldNames.mkString("[`", "`, `", "`]")}"))
}
- // Add the comment to a column, if comment is empty, return the original
column.
- private def addComment(column: StructField, comment: Option[String]):
StructField = {
- comment.map(column.withComment(_)).getOrElse(column)
- }
-
--- End diff --
Thanks for your advise!
I look into this in these days. With currently implement, all behavior
comply with Hive(Support type change/Work well in non binary format/Exception
in binary format like orc and parquet). Is it ok to add a config for constraint
this?
The work of adding logic to cast input data into changed type in catalog
may need modifying 4 parts logic including vectorized reader and row reader in
parquet and orc. If we don't agree the currently behavior, I'll keep following
these.
Item | Behavior
------------ | -------------
Parquet Row Reader | ClassCastException in SpecificInternalRow.set${Type}
Parquet Vectorized Reader | SchemaColumnConvertNotSupportedException in
VectorizedColumnReader.read${Type}Batch
Orc Row Reader | ClassCastException in OrcDeserializer.newWriter
Orc Vectorized Reader | NullPointerException in OrcColumnVector get value
by type method
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
