itsallsame opened a new issue, #4643: URL: https://github.com/apache/incubator-seatunnel/issues/4643
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened when use spark2 transfer data from hive to doris, get a error "java.lang.IllegalStateException: User did not initialize spark context! User did not initialize spark context!" btw, we use huawei cloud, and hive version is 3.1.0, data storage in Huawei Cloud Object Storage(OBS). ### SeaTunnel Version 2.3.1 ### SeaTunnel Config ```conf { "env": { "job.mode": "BATCH", "spark.yarn.queue": "data", "execution.parallelism": 1 }, "source": [ { "plugin_name": "Hive", "metastore_uri": "thrift://xxx:9083,thrift://xxx:9083", "table_name": "tmp.test_table" } ], "sink": [ { "plugin_name": "Console" } ] } ``` ### Running Command ```shell apache-seatunnel-incubating-2.3.1/bin/start-seatunnel-spark-2-connector-v2.sh \ --name test_seatunnle \ --check \ --deploy-mode cluster \ --master yarn \ --config config_test.json ``` ### Error Exception ```log 2023-04-21 17:48:56,479 | INFO | [main] | Starting the user application in a separate Thread | org.apache.spark.deploy.yarn.ApplicationMaster.logInfo(Logging.scala:54) 2023-04-21 17:48:56,506 | INFO | [main] | Waiting for spark context initialization... | org.apache.spark.deploy.yarn.ApplicationMaster.logInfo(Logging.scala:54) 2023-04-21 17:48:56,774 | INFO | [Driver] | Final app status: FAILED, exitCode: 13 | org.apache.spark.deploy.yarn.ApplicationMaster.logInfo(Logging.scala:54) 2023-04-21 17:48:56,778 | ERROR | [main] | Uncaught exception: | org.apache.spark.deploy.yarn.ApplicationMaster.logError(Logging.scala:91) java.lang.IllegalStateException: User did not initialize spark context! at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:492) at org.apache.spark.deploy.yarn.ApplicationMaster.org$apache$spark$deploy$yarn$ApplicationMaster$$runImpl(ApplicationMaster.scala:307) at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply$mcV$sp(ApplicationMaster.scala:247) at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply(ApplicationMaster.scala:247) at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply(ApplicationMaster.scala:247) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:802) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1761) at org.apache.spark.deploy.yarn.ApplicationMaster.doAsUser(ApplicationMaster.scala:801) at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:246) at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:828) at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala) ``` ### Flink or Spark Version spark2.4.5 ### Java or Scala Version jdk1.8 ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
