[
https://issues.apache.org/jira/browse/SPARK-40242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liang Fenjie updated SPARK-40242:
---------------------------------
Description:
Backgroud:
In industry development environment, we got used to write several spark-sql
blocks in one pyspark script. Without really submitting to cluster as running
application, getting all the physical plans of a application is too hard to do
within a parameter statement of "spark-submit" command.
Wish:
I wish to add a parameter "–genPlan" as for "spark-submit" command, which
can return all the physical plans of a application instead of submitting to run
indeed. Also, other approach to meet the issue is wellcome.
was:
Backgroud:
In industry development environment, we got used to write several spark-sql
blocks in one pyspark script. Without really submitting to cluster as running
application, getting all the physical plans of a application is too hard to do
within a parameter statement of "spark-submit" command.
Wish:
I wish to add a parameter "–genPlan" as for "spark-submit" command, which can
return all the physical plans of a application instead of submitting to run
indeed.
> Only return all physical plans after summitting pyspark script with several
> spark sql blocks inside
> ---------------------------------------------------------------------------------------------------
>
> Key: SPARK-40242
> URL: https://issues.apache.org/jira/browse/SPARK-40242
> Project: Spark
> Issue Type: Wish
> Components: Spark Submit, SQL
> Affects Versions: 2.1.3
> Reporter: Liang Fenjie
> Priority: Major
>
> Backgroud:
> In industry development environment, we got used to write several
> spark-sql blocks in one pyspark script. Without really submitting to cluster
> as running application, getting all the physical plans of a application is
> too hard to do within a parameter statement of "spark-submit" command.
>
> Wish:
> I wish to add a parameter "–genPlan" as for "spark-submit" command, which
> can return all the physical plans of a application instead of submitting to
> run indeed. Also, other approach to meet the issue is wellcome.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
