Serge Rielau created SPARK-47467:
------------------------------------
Summary: Error message regressed when creating hive table with
illegal column name
Key: SPARK-47467
URL: https://issues.apache.org/jira/browse/SPARK-47467
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 4.0.0
Reporter: Serge Rielau
The following statement used to result in:
CREATE TABLE test5(`北京, ` INT) USING HIVE;
[[INVALID_HIVE_COLUMN_NAME|https://docs.databricks.com/error-messages/error-classes.html#invalid_hive_column_name]]
Cannot create the table `hive_metastore`.`srielau`.`test5` having the column
`北京, ` whose name contains invalid characters ',' in Hive metastore. SQLSTATE:
42K05
Now it results in:
org.apache.spark.sql.AnalysisException:
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException:
MetaException(message:org.apache.hadoop.hive.serde2.SerDeException
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe: columns has 2 elements
while columns.types has 1 elements!) at
org.apache.spark.sql.hive.HiveExternalCatalog.$anonfun$withClient$2(HiveExternalCatalog.scala:168)
at
org.apache.spark.sql.hive.HiveExternalCatalog.maybeSynchronized(HiveExternalCatalog.scala:115)
at
org.apache.spark.sql.hive.HiveExternalCatalog.$anonfun$withClient$1(HiveExternalCatalog.scala:153)
at
com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressReporter.scala:405)
at
com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressReporter.scala:391)
at
com.databricks.spark.util.SparkDatabricksProgressReporter$.withStatusCode(ProgressReporter.scala:34)
at
org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:152)
at
org.apache.spark.sql.hive.HiveExternalCatalog.createTable(HiveExternalCatalog.scala:312)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.$anonfun$createTable$1(ExternalCatalogWithListener.scala:122)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at
org.apache.spark.sql.catalyst.MetricKeyUtils$.measure(MetricKey.scala:661) at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.$anonfun$profile$1(ExternalCatalogWithListener.scala:54)
at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94) at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.profile(ExternalCatalogWithListener.scala:53)
This may be related to:
https://github.com/apache/spark/pull/45180
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
