kevinshin created SPARK-50668:
---------------------------------
Summary: UNSUPPORTED_INSERT.RDD_BASED when insert into a jdbc
datasource
Key: SPARK-50668
URL: https://issues.apache.org/jira/browse/SPARK-50668
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.5.4
Environment: hdp 3.1.4
spark 3.5.4
Reporter: kevinshin
I have some SQL scripts for pushing the calculation results of the data
warehouse to the JDBC DATASOURCE.
When I attempted to upgrade from Spark 3.3.3 to Spark 3.5.4, I found that one
of them failed to run properly: it pushed data to the RDS database of Alibaba
Cloud, but other similar scripts worked without any issues.
This script can run normally on both Spark 3.3.3 and 3.4.4.
SCRIPT:
CREATE TEMPORARY VIEW test
USING org.apache.spark.sql.jdbc
OPTIONS (
url
"jdbc:mysql://rm-abc.mysql.rds.aliyuncs.com:3306/db_01?useUnicode=true&characterEncoding=UTF-8&useSSL=false",
dbtable "`apdata`",
user 'sgroot',
password '####'
);
explain INSERT INTO TABLE sg_apdata_v_prd select * from ods.ap_mapping_tmp;
== Physical Plan ==
org.apache.spark.sql.AnalysisException: [UNSUPPORTED_INSERT.RDD_BASED] Can't
insert into the target. An RDD-based table is not allowed.;
'InsertIntoStatement Project [user_name#0, region#1, ap_name#2, dtime#3,
staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils,
StringType, readSidePadding, ddate#4, 10, true, false, true) AS ddate#5],
false, fa
lse, false
+- Project [ap_name#58, addr#59]
+- SubqueryAlias spark_catalog.ods.ap_mapping_tmp
+- Relation spark_catalog.ods.ap_mapping_tmp[ap_name#58,addr#59] orc
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
