caican created SPARK-44414:
------------------------------
Summary: Fixed matching check for CharType/VarcharType
Key: SPARK-44414
URL: https://issues.apache.org/jira/browse/SPARK-44414
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.4.0, 3.3.0, 3.2.0, 3.1.2
Reporter: caican
Running the following code throws an exception
{code:java}
val analyzer = getAnalyzer
// check varchar type
val json1 = "{\"__CHAR_VARCHAR_TYPE_STRING\":\"varchar(80)\"}"
val metadata1 = new
MetadataBuilder().withMetadata(Metadata.fromJson(json1)).build()
val query1 = TestRelation(StructType(Seq(
StructField("x", StringType, metadata = metadata1),
StructField("y", StringType, metadata = metadata1))).toAttributes)
val table1 = TestRelation(StructType(Seq(
StructField("x", StringType, metadata = metadata1),
StructField("y", StringType, metadata = metadata1))).toAttributes)
val parsedPlanByName1 = byName(table1, query1)
analyzer.executeAndCheck(parsedPlanByName1, new QueryPlanningTracker()) {code}
Exception details are as follows
{code:java}
org.apache.spark.sql.AnalysisException: unresolved operator 'AppendData
TestRelation [x#8, y#9], true;
'AppendData TestRelation [x#8, y#9], true
+- TestRelation [x#6, y#7] at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis(CheckAnalysis.scala:52)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis$(CheckAnalysis.scala:51)
at
org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:156)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$47(CheckAnalysis.scala:704)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$47$adapted(CheckAnalysis.scala:702)
at
org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:186)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis.checkAnalysis(CheckAnalysis.scala:702)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis.checkAnalysis$(CheckAnalysis.scala:92)
at
org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:156)
at
org.apache.spark.sql.catalyst.analysis.Analyzer.$anonfun$executeAndCheck$1(Analyzer.scala:177)
at
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.markInAnalyzer(AnalysisHelper.scala:228)
at
org.apache.spark.sql.catalyst.analysis.Analyzer.executeAndCheck(Analyzer.scala:174)
at
org.apache.spark.sql.catalyst.analysis.DataSourceV2AnalysisBaseSuite.$anonfun$new$36(DataSourceV2AnalysisSuite.scala:691)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
