Jagadeesh Marada created SPARK-47489: ----------------------------------------
Summary: Action performed on Dataset is triggering an error as "The Spark SQL phase planning failed with an internal error." Key: SPARK-47489 URL: https://issues.apache.org/jira/browse/SPARK-47489 Project: Spark Issue Type: Bug Components: k8s, Spark Core, Spark Submit Affects Versions: 3.4.2 Reporter: Jagadeesh Marada Hi team, We are upgrading to spark 3.4.2 from 3.3.2 and facing issue in the spark driver as below. "org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase planning failed with an internal error. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace." Complete stack trace is as below. After analysing further , found that when we try to perform any actions on the dataset, spark is unable to plan its actions, results in exceptions. attaching a code snippet where exactly we are trigging an action to convert it as java RDD, is failing. Attaching the spark submit command as well for your reference. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org