[
https://issues.apache.org/jira/browse/SPARK-40093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17580069#comment-17580069
]
Hyukjin Kwon commented on SPARK-40093:
--------------------------------------
[~toopt4] the profile and release distributions are matched with our regular
Spark release. I think we should better raise a discussion in the mailing list
before filing it as a JIRA.
Short answer: I don't think we'll do this because such releases with the
exclusion is over-complicated but the benefit is rather trival (size of the
release)
> is kubernetes jar required if not using that executor?
> ------------------------------------------------------
>
> Key: SPARK-40093
> URL: https://issues.apache.org/jira/browse/SPARK-40093
> Project: Spark
> Issue Type: Question
> Components: Deploy
> Affects Versions: 3.3.0
> Reporter: t oo
> Priority: Major
>
> my docker file is very big with pyspark
> can i remove dis files below if i don't use 'kubernetes executor'?
> 11M total
> 4.0M
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-core-5.12.2.jar
> 840K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-client-5.12.2.jar
> 760K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-admissionregistration-5.12.2.jar
> 704K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-apiextensions-5.12.2.jar
> 640K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-autoscaling-5.12.2.jar
> 528K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-extensions-5.12.2.jar
> 516K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/spark-kubernetes_2.12-3.3.0.jar
> 456K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-networking-5.12.2.jar
> 436K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-apps-5.12.2.jar
> 364K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-storageclass-5.12.2.jar
> 336K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-policy-5.12.2.jar
> 264K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-flowcontrol-5.12.2.jar
> 244K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-batch-5.12.2.jar
> 192K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-discovery-5.12.2.jar
> 176K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-rbac-5.12.2.jar
> 160K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-node-5.12.2.jar
> 144K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-certificates-5.12.2.jar
> 104K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-events-5.12.2.jar
> 80K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-metrics-5.12.2.jar
> 68K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-scheduling-5.12.2.jar
> 48K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-coordination-5.12.2.jar
> 20K
> /usr/local/lib/python3.10/site-packages/pyspark/jars/kubernetes-model-common-5.12.2.jar
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
