[jira] [Resolved] (SPARK-41257) Upgrade actions/labeler to v4
[ https://issues.apache.org/jira/browse/SPARK-41257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-41257. - Resolution: Fixed > Upgrade actions/labeler to v4 > - > > Key: SPARK-41257 > URL: https://issues.apache.org/jira/browse/SPARK-41257 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-41257) Upgrade actions/labeler to v4
[ https://issues.apache.org/jira/browse/SPARK-41257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-41257: --- Assignee: Yikun Jiang > Upgrade actions/labeler to v4 > - > > Key: SPARK-41257 > URL: https://issues.apache.org/jira/browse/SPARK-41257 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-41257) Upgrade actions/labeler to v4
Yikun Jiang created SPARK-41257: --- Summary: Upgrade actions/labeler to v4 Key: SPARK-41257 URL: https://issues.apache.org/jira/browse/SPARK-41257 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-41251) Upgrade pandas from 1.5.1 to 1.5.2
[ https://issues.apache.org/jira/browse/SPARK-41251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-41251. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38787 [https://github.com/apache/spark/pull/38787] > Upgrade pandas from 1.5.1 to 1.5.2 > -- > > Key: SPARK-41251 > URL: https://issues.apache.org/jira/browse/SPARK-41251 > Project: Spark > Issue Type: Improvement > Components: Project Infra, ps >Affects Versions: 3.4.0 >Reporter: BingKun Pan >Assignee: BingKun Pan >Priority: Minor > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-41251) Upgrade pandas from 1.5.1 to 1.5.2
[ https://issues.apache.org/jira/browse/SPARK-41251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-41251: --- Assignee: BingKun Pan > Upgrade pandas from 1.5.1 to 1.5.2 > -- > > Key: SPARK-41251 > URL: https://issues.apache.org/jira/browse/SPARK-41251 > Project: Spark > Issue Type: Improvement > Components: Project Infra, ps >Affects Versions: 3.4.0 >Reporter: BingKun Pan >Assignee: BingKun Pan >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-41253) Make K8s volcano IT work in Github Action
Yikun Jiang created SPARK-41253: --- Summary: Make K8s volcano IT work in Github Action Key: SPARK-41253 URL: https://issues.apache.org/jira/browse/SPARK-41253 Project: Spark Issue Type: Bug Components: Kubernetes Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-41185) YuniKorn on ARM incorrectly documented as not supported
[ https://issues.apache.org/jira/browse/SPARK-41185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637041#comment-17637041 ] Yikun Jiang edited comment on SPARK-41185 at 11/22/22 7:36 AM: --- Yes, I think arm64 support already introduced since v1.1 in Apache Yunikorn. The doc was for yunikorn v1.0 I guess. cc [~dongjoon] Maybe we should update doc? See related in [1] [1] https://github.com/apache/spark/pull/37872 was (Author: yikunkero): Yes, I think arm64 support already introduced since v1.1 in Apache Yunikorn. cc [~dongjoon] Maybe we should update doc? See related in [1] [1] https://github.com/apache/spark/pull/37872 > YuniKorn on ARM incorrectly documented as not supported > --- > > Key: SPARK-41185 > URL: https://issues.apache.org/jira/browse/SPARK-41185 > Project: Spark > Issue Type: Documentation > Components: Documentation >Affects Versions: 3.3.1 >Reporter: Wilfred Spiegelenburg >Priority: Major > > YuniKorn does support ARM. There is no limitation. YuniKorn 1.1 fully > supports deploying and running on ARM. Multi architecture images are > published as part of each release. > The limitation mentioned in the documentation that YuniKorn does not support > ARM is incorrect and should be removed: > https://spark.apache.org/docs/3.3.1/running-on-kubernetes.html#limitations -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41185) YuniKorn on ARM incorrectly documented as not supported
[ https://issues.apache.org/jira/browse/SPARK-41185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637041#comment-17637041 ] Yikun Jiang commented on SPARK-41185: - Yes, I think arm64 support already introduced since v1.1 in Apache Yunikorn. cc [~dongjoon] Maybe we should update doc? See related in [1] [1] https://github.com/apache/spark/pull/37872 > YuniKorn on ARM incorrectly documented as not supported > --- > > Key: SPARK-41185 > URL: https://issues.apache.org/jira/browse/SPARK-41185 > Project: Spark > Issue Type: Documentation > Components: Documentation >Affects Versions: 3.3.1 >Reporter: Wilfred Spiegelenburg >Priority: Major > > YuniKorn does support ARM. There is no limitation. YuniKorn 1.1 fully > supports deploying and running on ARM. Multi architecture images are > published as part of each release. > The limitation mentioned in the documentation that YuniKorn does not support > ARM is incorrect and should be removed: > https://spark.apache.org/docs/3.3.1/running-on-kubernetes.html#limitations -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-41186) Fix doctest for new version mlfow
[
https://issues.apache.org/jira/browse/SPARK-41186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang resolved SPARK-41186.
-
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 38698
[https://github.com/apache/spark/pull/38698]
> Fix doctest for new version mlfow
> -
>
> Key: SPARK-41186
> URL: https://issues.apache.org/jira/browse/SPARK-41186
> Project: Spark
> Issue Type: Bug
> Components: Tests
>Affects Versions: 3.4.0
>Reporter: Yikun Jiang
>Assignee: Yikun Jiang
>Priority: Major
> Fix For: 3.4.0
>
>
>
>
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 168, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> run_info = client.list_run_infos(exp_id)[-1]
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> run_info = client.list_run_infos(exp_id)[-1]
> AttributeError: 'MlflowClient' object has no attribute 'list_run_infos'
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 169, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> model =
> load_model("runs:/{run_id}/model".format(run_id=run_info.run_uuid))
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> model =
> load_model("runs:/{run_id}/model".format(run_id=run_info.run_uuid))
> NameError: name 'run_info' is not defined
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 171, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> prediction_df["prediction"] = model.predict(prediction_df)
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> prediction_df["prediction"] = model.predict(prediction_df)
> NameError: name 'model' is not defined
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 172, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> prediction_df
> Expected:
> x1 x2 prediction
> 0 2.0 4.01.31
> Got:
> x1 x2
> 0 2.0 4.0
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 178, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> model.predict(prediction_df[["x1", "x2"]].to_pandas())
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> model.predict(prediction_df[["x1", "x2"]].to_pandas())
> NameError: name 'model' is not defined
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 189, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> y = model.predict(features)
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> y = model.predict(features)
> NameError: name 'model' is not defined
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 198, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> features['y'] = y
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> features['y'] = y
> NameError: name 'y' is not defined
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 200, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> everything
> Expected:
> x1 x2 z y
> 0 2.0 3.0 -1 1.376932
> Got:
> x1 x2 z
> 0 2.0 3.0 -1
>
[jira] [Assigned] (SPARK-41186) Fix doctest for new version mlfow
[
https://issues.apache.org/jira/browse/SPARK-41186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang reassigned SPARK-41186:
---
Assignee: Yikun Jiang
> Fix doctest for new version mlfow
> -
>
> Key: SPARK-41186
> URL: https://issues.apache.org/jira/browse/SPARK-41186
> Project: Spark
> Issue Type: Bug
> Components: Tests
>Affects Versions: 3.4.0
>Reporter: Yikun Jiang
>Assignee: Yikun Jiang
>Priority: Major
>
>
>
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 168, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> run_info = client.list_run_infos(exp_id)[-1]
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> run_info = client.list_run_infos(exp_id)[-1]
> AttributeError: 'MlflowClient' object has no attribute 'list_run_infos'
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 169, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> model =
> load_model("runs:/{run_id}/model".format(run_id=run_info.run_uuid))
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> model =
> load_model("runs:/{run_id}/model".format(run_id=run_info.run_uuid))
> NameError: name 'run_info' is not defined
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 171, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> prediction_df["prediction"] = model.predict(prediction_df)
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> prediction_df["prediction"] = model.predict(prediction_df)
> NameError: name 'model' is not defined
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 172, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> prediction_df
> Expected:
> x1 x2 prediction
> 0 2.0 4.01.31
> Got:
> x1 x2
> 0 2.0 4.0
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 178, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> model.predict(prediction_df[["x1", "x2"]].to_pandas())
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> model.predict(prediction_df[["x1", "x2"]].to_pandas())
> NameError: name 'model' is not defined
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 189, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> y = model.predict(features)
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> y = model.predict(features)
> NameError: name 'model' is not defined
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 198, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> features['y'] = y
> Exception raised:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/doctest.py", line 1336, in __run
> exec(compile(example.source, filename, "single",
> File "", line 1, in
>
> features['y'] = y
> NameError: name 'y' is not defined
> **
> File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 200, in
> pyspark.pandas.mlflow.load_model
> Failed example:
> everything
> Expected:
> x1 x2 z y
> 0 2.0 3.0 -1 1.376932
> Got:
> x1 x2 z
> 0 2.0 3.0 -1
> **
>8 of 26 in pyspark.pandas.mlflow.load_model
--
This message
[jira] [Assigned] (SPARK-41107) Install memory-profiler in the CI
[ https://issues.apache.org/jira/browse/SPARK-41107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-41107: --- Assignee: Xinrong Meng > Install memory-profiler in the CI > - > > Key: SPARK-41107 > URL: https://issues.apache.org/jira/browse/SPARK-41107 > Project: Spark > Issue Type: Sub-task > Components: Build, PySpark, Tests >Affects Versions: 3.4.0 >Reporter: Xinrong Meng >Assignee: Xinrong Meng >Priority: Major > > PySpark memory profiler depends on > [memory-profiler|https://pypi.org/project/memory-profiler/]. > The ticket proposes to install memory-profiler in the CI to enable related > tests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-41107) Install memory-profiler in the CI
[ https://issues.apache.org/jira/browse/SPARK-41107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-41107. - Resolution: Fixed Resolved by https://github.com/apache/spark/pull/38611 > Install memory-profiler in the CI > - > > Key: SPARK-41107 > URL: https://issues.apache.org/jira/browse/SPARK-41107 > Project: Spark > Issue Type: Sub-task > Components: Build, PySpark, Tests >Affects Versions: 3.4.0 >Reporter: Xinrong Meng >Assignee: Xinrong Meng >Priority: Major > > PySpark memory profiler depends on > [memory-profiler|https://pypi.org/project/memory-profiler/]. > The ticket proposes to install memory-profiler in the CI to enable related > tests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-41186) Fix doctest for new version mlfow
Yikun Jiang created SPARK-41186:
---
Summary: Fix doctest for new version mlfow
Key: SPARK-41186
URL: https://issues.apache.org/jira/browse/SPARK-41186
Project: Spark
Issue Type: Bug
Components: Tests
Affects Versions: 3.4.0
Reporter: Yikun Jiang
**
File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 168, in
pyspark.pandas.mlflow.load_model
Failed example:
run_info = client.list_run_infos(exp_id)[-1]
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.9/doctest.py", line 1336, in __run
exec(compile(example.source, filename, "single",
File "", line 1, in
run_info = client.list_run_infos(exp_id)[-1]
AttributeError: 'MlflowClient' object has no attribute 'list_run_infos'
**
File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 169, in
pyspark.pandas.mlflow.load_model
Failed example:
model = load_model("runs:/{run_id}/model".format(run_id=run_info.run_uuid))
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.9/doctest.py", line 1336, in __run
exec(compile(example.source, filename, "single",
File "", line 1, in
model =
load_model("runs:/{run_id}/model".format(run_id=run_info.run_uuid))
NameError: name 'run_info' is not defined
**
File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 171, in
pyspark.pandas.mlflow.load_model
Failed example:
prediction_df["prediction"] = model.predict(prediction_df)
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.9/doctest.py", line 1336, in __run
exec(compile(example.source, filename, "single",
File "", line 1, in
prediction_df["prediction"] = model.predict(prediction_df)
NameError: name 'model' is not defined
**
File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 172, in
pyspark.pandas.mlflow.load_model
Failed example:
prediction_df
Expected:
x1 x2 prediction
0 2.0 4.01.31
Got:
x1 x2
0 2.0 4.0
**
File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 178, in
pyspark.pandas.mlflow.load_model
Failed example:
model.predict(prediction_df[["x1", "x2"]].to_pandas())
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.9/doctest.py", line 1336, in __run
exec(compile(example.source, filename, "single",
File "", line 1, in
model.predict(prediction_df[["x1", "x2"]].to_pandas())
NameError: name 'model' is not defined
**
File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 189, in
pyspark.pandas.mlflow.load_model
Failed example:
y = model.predict(features)
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.9/doctest.py", line 1336, in __run
exec(compile(example.source, filename, "single",
File "", line 1, in
y = model.predict(features)
NameError: name 'model' is not defined
**
File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 198, in
pyspark.pandas.mlflow.load_model
Failed example:
features['y'] = y
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.9/doctest.py", line 1336, in __run
exec(compile(example.source, filename, "single",
File "", line 1, in
features['y'] = y
NameError: name 'y' is not defined
**
File "/__w/spark/spark/python/pyspark/pandas/mlflow.py", line 200, in
pyspark.pandas.mlflow.load_model
Failed example:
everything
Expected:
x1 x2 z y
0 2.0 3.0 -1 1.376932
Got:
x1 x2 z
0 2.0 3.0 -1
**
8 of 26 in pyspark.pandas.mlflow.load_model
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-41075) Can't build Spark docker image with Java 17
[
https://issues.apache.org/jira/browse/SPARK-41075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635071#comment-17635071
]
Yikun Jiang edited comment on SPARK-41075 at 11/17/22 1:56 AM:
---
I think the `kubernetes/dockerfiles/spark/` is the note for binary path, like
```
wget https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz
tar -zxvf spark-3.3.1-bin-hadoop3.tgz
cd spark-3.3.1-bin-hadoop3\
./bin/docker-image-tool.sh -r xxx -t 3.3.1 -f
kubernetes/dockerfiles/spark/Dockerfile.java17 build
```
But if you are using the source code, you should set the correct path like
resource-managers/kubernetes/docker/src/main/dockerfiles.
BTW, below info you might interested:
- After 3.4.0, java17 dockerfiles is default Dockerfile in k8s IT:
https://github.com/apache/spark/pull/38417
- We create a spark-docker to help build image simpler:
https://github.com/apache/spark-docker
was (Author: yikunkero):
I think the `kubernetes/dockerfiles/spark/` is for binary path.
```
wget https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz
tar -zxvf spark-3.3.1-bin-hadoop3.tgz
cd spark-3.3.1-bin-hadoop3\
./bin/docker-image-tool.sh -r xxx -t 3.3.1 -f
kubernetes/dockerfiles/spark/Dockerfile.java17 build
```
But if you are using the source code, you should set the correct path like
resource-managers/kubernetes/docker/src/main/dockerfiles.
BTW, below info you might interested:
- After 3.4.0, java17 dockerfiles is default Dockerfile in k8s IT:
https://github.com/apache/spark/pull/38417
- We create a spark-docker to help build image simpler:
https://github.com/apache/spark-docker
> Can't build Spark docker image with Java 17
> ---
>
> Key: SPARK-41075
> URL: https://issues.apache.org/jira/browse/SPARK-41075
> Project: Spark
> Issue Type: Bug
> Components: Build
>Affects Versions: 3.3.1
>Reporter: Yauheni Shahun
>Priority: Minor
>
> I'm trying to build Spark docker image with Java 17 according to usage
> description from {{{}docker-image-tool.sh{}}}:
> {code:java}
> ./bin/docker-image-tool.sh -r $REGISTRY -t ${VERSION} -f
> kubernetes/dockerfiles/spark/Dockerfile.java17 build
> {code}
> But I get the following error:
> {code:java}
> ./bin/docker-image-tool.sh: line 74: cd: kubernetes/dockerfiles/spark: No
> such file or directory
> {code}
> As far as I understand the script verifies the argument for presence before
> it creates a build context and copies content from
> {{{}resource-managers/kubernetes/docker/src/main/dockerfiles{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41075) Can't build Spark docker image with Java 17
[
https://issues.apache.org/jira/browse/SPARK-41075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635071#comment-17635071
]
Yikun Jiang commented on SPARK-41075:
-
I think the `kubernetes/dockerfiles/spark/` is for binary path.
```
wget https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz
tar -zxvf spark-3.3.1-bin-hadoop3.tgz
cd spark-3.3.1-bin-hadoop3\
./bin/docker-image-tool.sh -r xxx -t 3.3.1 -f
kubernetes/dockerfiles/spark/Dockerfile.java17 build
```
But if you are using the source code, you should set the correct path like
resource-managers/kubernetes/docker/src/main/dockerfiles.
BTW, below info you might interested:
- After 3.4.0, java17 dockerfiles is default Dockerfile in k8s IT:
https://github.com/apache/spark/pull/38417
- We create a spark-docker to help build image simpler:
https://github.com/apache/spark-docker
> Can't build Spark docker image with Java 17
> ---
>
> Key: SPARK-41075
> URL: https://issues.apache.org/jira/browse/SPARK-41075
> Project: Spark
> Issue Type: Bug
> Components: Build
>Affects Versions: 3.3.1
>Reporter: Yauheni Shahun
>Priority: Minor
>
> I'm trying to build Spark docker image with Java 17 according to usage
> description from {{{}docker-image-tool.sh{}}}:
> {code:java}
> ./bin/docker-image-tool.sh -r $REGISTRY -t ${VERSION} -f
> kubernetes/dockerfiles/spark/Dockerfile.java17 build
> {code}
> But I get the following error:
> {code:java}
> ./bin/docker-image-tool.sh: line 74: cd: kubernetes/dockerfiles/spark: No
> such file or directory
> {code}
> As far as I understand the script verifies the argument for presence before
> it creates a build context and copies content from
> {{{}resource-managers/kubernetes/docker/src/main/dockerfiles{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40519) Add "Publish workflow" to help release apache/spark image
[ https://issues.apache.org/jira/browse/SPARK-40519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40519. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 23 [https://github.com/apache/spark-docker/pull/23] > Add "Publish workflow" to help release apache/spark image > - > > Key: SPARK-40519 > URL: https://issues.apache.org/jira/browse/SPARK-40519 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > > Example > [https://github.com/Yikun/spark-docker/blob/master/.github/workflows/publish.yaml] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40513) SPIP: Support Docker Official Image for Spark
[ https://issues.apache.org/jira/browse/SPARK-40513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17630735#comment-17630735 ] Yikun Jiang commented on SPARK-40513: - Enable Github Autolink references for spark-docker https://issues.apache.org/jira/browse/INFRA-23789 > SPIP: Support Docker Official Image for Spark > - > > Key: SPARK-40513 > URL: https://issues.apache.org/jira/browse/SPARK-40513 > Project: Spark > Issue Type: New Feature > Components: Kubernetes, PySpark, SparkR >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Priority: Major > Labels: SPIP > > This SPIP is proposed to add [Docker Official > Image(DOI)|https://github.com/docker-library/official-images] to ensure the > Spark Docker images meet the quality standards for Docker images, to provide > these Docker images for users who want to use Apache Spark via Docker image. > There are also several [Apache projects that release the Docker Official > Images|https://hub.docker.com/search?q=apache_filter=official], such > as: [flink|https://hub.docker.com/_/flink], > [storm|https://hub.docker.com/_/storm], [solr|https://hub.docker.com/_/solr], > [zookeeper|https://hub.docker.com/_/zookeeper], > [httpd|https://hub.docker.com/_/httpd] (with 50M+ to 1B+ download for each). > From the huge download statistics, we can see the real demands of users, and > from the support of other apache projects, we should also be able to do it. > After support: > * The Dockerfile will still be maintained by the Apache Spark community and > reviewed by Docker. > * The images will be maintained by the Docker community to ensure the > quality standards for Docker images of the Docker community. > It will also reduce the extra docker images maintenance effort (such as > frequently rebuilding, image security update) of the Apache Spark community. > > SPIP DOC: > [https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o] > DISCUSS: [https://lists.apache.org/thread/l1793y5224n8bqkp3s6ltgkykso4htb3] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40513) SPIP: Support Docker Official Image for Spark
[ https://issues.apache.org/jira/browse/SPARK-40513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17630733#comment-17630733 ] Yikun Jiang commented on SPARK-40513: - Add secrets.DOCKER_USER and secrets.DOCKER_TOKEN for apache/spark-docker https://issues.apache.org/jira/browse/INFRA-23882 > SPIP: Support Docker Official Image for Spark > - > > Key: SPARK-40513 > URL: https://issues.apache.org/jira/browse/SPARK-40513 > Project: Spark > Issue Type: New Feature > Components: Kubernetes, PySpark, SparkR >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Priority: Major > Labels: SPIP > > This SPIP is proposed to add [Docker Official > Image(DOI)|https://github.com/docker-library/official-images] to ensure the > Spark Docker images meet the quality standards for Docker images, to provide > these Docker images for users who want to use Apache Spark via Docker image. > There are also several [Apache projects that release the Docker Official > Images|https://hub.docker.com/search?q=apache_filter=official], such > as: [flink|https://hub.docker.com/_/flink], > [storm|https://hub.docker.com/_/storm], [solr|https://hub.docker.com/_/solr], > [zookeeper|https://hub.docker.com/_/zookeeper], > [httpd|https://hub.docker.com/_/httpd] (with 50M+ to 1B+ download for each). > From the huge download statistics, we can see the real demands of users, and > from the support of other apache projects, we should also be able to do it. > After support: > * The Dockerfile will still be maintained by the Apache Spark community and > reviewed by Docker. > * The images will be maintained by the Docker community to ensure the > quality standards for Docker images of the Docker community. > It will also reduce the extra docker images maintenance effort (such as > frequently rebuilding, image security update) of the Apache Spark community. > > SPIP DOC: > [https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o] > DISCUSS: [https://lists.apache.org/thread/l1793y5224n8bqkp3s6ltgkykso4htb3] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40569) Add smoke test in standalone cluster for spark-docker
[ https://issues.apache.org/jira/browse/SPARK-40569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40569. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 21 [https://github.com/apache/spark-docker/pull/21] > Add smoke test in standalone cluster for spark-docker > - > > Key: SPARK-40569 > URL: https://issues.apache.org/jira/browse/SPARK-40569 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Qian Sun >Assignee: Qian Sun >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40969) Unable to download spark 3.3.0 tarball after 3.3.1 release in spark-docker
[
https://issues.apache.org/jira/browse/SPARK-40969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang resolved SPARK-40969.
-
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 22
[https://github.com/apache/spark-docker/pull/22]
> Unable to download spark 3.3.0 tarball after 3.3.1 release in spark-docker
> --
>
> Key: SPARK-40969
> URL: https://issues.apache.org/jira/browse/SPARK-40969
> Project: Spark
> Issue Type: Bug
> Components: Spark Docker
>Affects Versions: 3.3.1
>Reporter: Qian Sun
>Assignee: Qian Sun
>Priority: Major
> Fix For: 3.4.0
>
>
> Unable to download spark 3.3.0 tarball in spark-docker.
> {code:sh}
> #7 0.229 + wget -nv -O spark.tgz
> https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz
> #7 1.061
> https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz:
> #7 1.061 2022-10-31 02:59:20 ERROR 404: Not Found.
> --
> executor failed running [/bin/sh -c set -ex; export SPARK_TMP="$(mktemp
> -d)"; cd $SPARK_TMP; wget -nv -O spark.tgz "$SPARK_TGZ_URL"; wget
> -nv -O spark.tgz.asc "$SPARK_TGZ_ASC_URL"; export GNUPGHOME="$(mktemp
> -d)"; gpg --keyserver hkps://keys.openpgp.org --recv-key "$GPG_KEY" ||
> gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys "$GPG_KEY"; gpg
> --batch --verify spark.tgz.asc spark.tgz; gpgconf --kill all; rm -rf
> "$GNUPGHOME" spark.tgz.asc; tar -xf spark.tgz --strip-components=1;
> chown -R spark:spark .; mv jars /opt/spark/; mv bin /opt/spark/;
> mv sbin /opt/spark/; mv kubernetes/dockerfiles/spark/decom.sh /opt/;
> mv examples /opt/spark/; mv kubernetes/tests /opt/spark/; mv data
> /opt/spark/; mv python/pyspark /opt/spark/python/pyspark/; mv
> python/lib /opt/spark/python/lib/; cd ..; rm -rf "$SPARK_TMP";]: exit
> code: 8
> {code}
> And spark 3.3.1 docker is ok
> {code:sh}
> => [4/9] RUN set -ex; export SPARK_TMP="$(mktemp -d)"; cd $SPARK_TMP;
> wget -nv -O spark.tgz
> "https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz;;
> wget -nv -O spark.tgz.asc "https://downlo 77.8s
> => [5/9] COPY entrypoint.sh /opt/
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40969) Unable to download spark 3.3.0 tarball after 3.3.1 release in spark-docker
[
https://issues.apache.org/jira/browse/SPARK-40969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang reassigned SPARK-40969:
---
Assignee: Qian Sun
> Unable to download spark 3.3.0 tarball after 3.3.1 release in spark-docker
> --
>
> Key: SPARK-40969
> URL: https://issues.apache.org/jira/browse/SPARK-40969
> Project: Spark
> Issue Type: Bug
> Components: Spark Docker
>Affects Versions: 3.3.1
>Reporter: Qian Sun
>Assignee: Qian Sun
>Priority: Major
>
> Unable to download spark 3.3.0 tarball in spark-docker.
> {code:sh}
> #7 0.229 + wget -nv -O spark.tgz
> https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz
> #7 1.061
> https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz:
> #7 1.061 2022-10-31 02:59:20 ERROR 404: Not Found.
> --
> executor failed running [/bin/sh -c set -ex; export SPARK_TMP="$(mktemp
> -d)"; cd $SPARK_TMP; wget -nv -O spark.tgz "$SPARK_TGZ_URL"; wget
> -nv -O spark.tgz.asc "$SPARK_TGZ_ASC_URL"; export GNUPGHOME="$(mktemp
> -d)"; gpg --keyserver hkps://keys.openpgp.org --recv-key "$GPG_KEY" ||
> gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys "$GPG_KEY"; gpg
> --batch --verify spark.tgz.asc spark.tgz; gpgconf --kill all; rm -rf
> "$GNUPGHOME" spark.tgz.asc; tar -xf spark.tgz --strip-components=1;
> chown -R spark:spark .; mv jars /opt/spark/; mv bin /opt/spark/;
> mv sbin /opt/spark/; mv kubernetes/dockerfiles/spark/decom.sh /opt/;
> mv examples /opt/spark/; mv kubernetes/tests /opt/spark/; mv data
> /opt/spark/; mv python/pyspark /opt/spark/python/pyspark/; mv
> python/lib /opt/spark/python/lib/; cd ..; rm -rf "$SPARK_TMP";]: exit
> code: 8
> {code}
> And spark 3.3.1 docker is ok
> {code:sh}
> => [4/9] RUN set -ex; export SPARK_TMP="$(mktemp -d)"; cd $SPARK_TMP;
> wget -nv -O spark.tgz
> "https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz;;
> wget -nv -O spark.tgz.asc "https://downlo 77.8s
> => [5/9] COPY entrypoint.sh /opt/
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-40969) Unable to download spark 3.3.0 tarball after 3.3.1 release in spark-docker
[
https://issues.apache.org/jira/browse/SPARK-40969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17626358#comment-17626358
]
Yikun Jiang edited comment on SPARK-40969 at 10/31/22 3:25 AM:
---
Let's use the https://archive.apache.org/dist/spark/ , mind to submit a PR?
Thanks
was (Author: yikunkero):
Let's use the https://archive.apache.org/dist/spark/
> Unable to download spark 3.3.0 tarball after 3.3.1 release in spark-docker
> --
>
> Key: SPARK-40969
> URL: https://issues.apache.org/jira/browse/SPARK-40969
> Project: Spark
> Issue Type: Bug
> Components: Spark Docker
>Affects Versions: 3.3.1
>Reporter: Qian Sun
>Priority: Major
>
> Unable to download spark 3.3.0 tarball in spark-docker.
> {code:sh}
> #7 0.229 + wget -nv -O spark.tgz
> https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz
> #7 1.061
> https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz:
> #7 1.061 2022-10-31 02:59:20 ERROR 404: Not Found.
> --
> executor failed running [/bin/sh -c set -ex; export SPARK_TMP="$(mktemp
> -d)"; cd $SPARK_TMP; wget -nv -O spark.tgz "$SPARK_TGZ_URL"; wget
> -nv -O spark.tgz.asc "$SPARK_TGZ_ASC_URL"; export GNUPGHOME="$(mktemp
> -d)"; gpg --keyserver hkps://keys.openpgp.org --recv-key "$GPG_KEY" ||
> gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys "$GPG_KEY"; gpg
> --batch --verify spark.tgz.asc spark.tgz; gpgconf --kill all; rm -rf
> "$GNUPGHOME" spark.tgz.asc; tar -xf spark.tgz --strip-components=1;
> chown -R spark:spark .; mv jars /opt/spark/; mv bin /opt/spark/;
> mv sbin /opt/spark/; mv kubernetes/dockerfiles/spark/decom.sh /opt/;
> mv examples /opt/spark/; mv kubernetes/tests /opt/spark/; mv data
> /opt/spark/; mv python/pyspark /opt/spark/python/pyspark/; mv
> python/lib /opt/spark/python/lib/; cd ..; rm -rf "$SPARK_TMP";]: exit
> code: 8
> {code}
> And spark 3.3.1 docker is ok
> {code:sh}
> => [4/9] RUN set -ex; export SPARK_TMP="$(mktemp -d)"; cd $SPARK_TMP;
> wget -nv -O spark.tgz
> "https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz;;
> wget -nv -O spark.tgz.asc "https://downlo 77.8s
> => [5/9] COPY entrypoint.sh /opt/
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40969) Unable to download spark 3.3.0 tarball after 3.3.1 release in spark-docker
[
https://issues.apache.org/jira/browse/SPARK-40969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17626358#comment-17626358
]
Yikun Jiang commented on SPARK-40969:
-
Let's use the https://archive.apache.org/dist/spark/
> Unable to download spark 3.3.0 tarball after 3.3.1 release in spark-docker
> --
>
> Key: SPARK-40969
> URL: https://issues.apache.org/jira/browse/SPARK-40969
> Project: Spark
> Issue Type: Bug
> Components: Spark Docker
>Affects Versions: 3.3.1
>Reporter: Qian Sun
>Priority: Major
>
> Unable to download spark 3.3.0 tarball in spark-docker.
> {code:sh}
> #7 0.229 + wget -nv -O spark.tgz
> https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz
> #7 1.061
> https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz:
> #7 1.061 2022-10-31 02:59:20 ERROR 404: Not Found.
> --
> executor failed running [/bin/sh -c set -ex; export SPARK_TMP="$(mktemp
> -d)"; cd $SPARK_TMP; wget -nv -O spark.tgz "$SPARK_TGZ_URL"; wget
> -nv -O spark.tgz.asc "$SPARK_TGZ_ASC_URL"; export GNUPGHOME="$(mktemp
> -d)"; gpg --keyserver hkps://keys.openpgp.org --recv-key "$GPG_KEY" ||
> gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys "$GPG_KEY"; gpg
> --batch --verify spark.tgz.asc spark.tgz; gpgconf --kill all; rm -rf
> "$GNUPGHOME" spark.tgz.asc; tar -xf spark.tgz --strip-components=1;
> chown -R spark:spark .; mv jars /opt/spark/; mv bin /opt/spark/;
> mv sbin /opt/spark/; mv kubernetes/dockerfiles/spark/decom.sh /opt/;
> mv examples /opt/spark/; mv kubernetes/tests /opt/spark/; mv data
> /opt/spark/; mv python/pyspark /opt/spark/python/pyspark/; mv
> python/lib /opt/spark/python/lib/; cd ..; rm -rf "$SPARK_TMP";]: exit
> code: 8
> {code}
> And spark 3.3.1 docker is ok
> {code:sh}
> => [4/9] RUN set -ex; export SPARK_TMP="$(mktemp -d)"; cd $SPARK_TMP;
> wget -nv -O spark.tgz
> "https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz;;
> wget -nv -O spark.tgz.asc "https://downlo 77.8s
> => [5/9] COPY entrypoint.sh /opt/
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40858) Cleanup github action warning
[ https://issues.apache.org/jira/browse/SPARK-40858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17624965#comment-17624965 ] Yikun Jiang commented on SPARK-40858: - All warning cleanup: https://github.com/apache/spark/actions/runs/3334910929 Resolved. > Cleanup github action warning > - > > Key: SPARK-40858 > URL: https://issues.apache.org/jira/browse/SPARK-40858 > Project: Spark > Issue Type: Bug > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > > * The `save-state` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * The `set-output` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * Node.js 12 actions are deprecated. For more information see: > https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. > Please update the following actions to use Node.js 16: actions/checkout, > docker/setup-qemu-action, docker/setup-buildx-action, > docker/build-push-action, docker/build-push-action, > docker/setup-buildx-action, actions/checkout > * > like: https://github.com/apache/spark/actions/runs/3288503290 > In order to reduce pontential impact on infra, we'd better address them one > by one according actions type. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] (SPARK-40858) Cleanup github action warning
[ https://issues.apache.org/jira/browse/SPARK-40858 ] Yikun Jiang deleted comment on SPARK-40858: - was (Author: hyukjin.kwon): Issue resolved by pull request 38408 [https://github.com/apache/spark/pull/38408] > Cleanup github action warning > - > > Key: SPARK-40858 > URL: https://issues.apache.org/jira/browse/SPARK-40858 > Project: Spark > Issue Type: Bug > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > > * The `save-state` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * The `set-output` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * Node.js 12 actions are deprecated. For more information see: > https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. > Please update the following actions to use Node.js 16: actions/checkout, > docker/setup-qemu-action, docker/setup-buildx-action, > docker/build-push-action, docker/build-push-action, > docker/setup-buildx-action, actions/checkout > * > like: https://github.com/apache/spark/actions/runs/3288503290 > In order to reduce pontential impact on infra, we'd better address them one > by one according actions type. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] (SPARK-40858) Cleanup github action warning
[ https://issues.apache.org/jira/browse/SPARK-40858 ] Yikun Jiang deleted comment on SPARK-40858: - was (Author: apachespark): User 'Yikun' has created a pull request for this issue: https://github.com/apache/spark/pull/38408 > Cleanup github action warning > - > > Key: SPARK-40858 > URL: https://issues.apache.org/jira/browse/SPARK-40858 > Project: Spark > Issue Type: Bug > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > > * The `save-state` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * The `set-output` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * Node.js 12 actions are deprecated. For more information see: > https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. > Please update the following actions to use Node.js 16: actions/checkout, > docker/setup-qemu-action, docker/setup-buildx-action, > docker/build-push-action, docker/build-push-action, > docker/setup-buildx-action, actions/checkout > * > like: https://github.com/apache/spark/actions/runs/3288503290 > In order to reduce pontential impact on infra, we'd better address them one > by one according actions type. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40928) Upgrade actions/setup-python to v4
[ https://issues.apache.org/jira/browse/SPARK-40928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40928: --- Assignee: Yikun Jiang > Upgrade actions/setup-python to v4 > -- > > Key: SPARK-40928 > URL: https://issues.apache.org/jira/browse/SPARK-40928 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] (SPARK-40858) Cleanup github action warning
[ https://issues.apache.org/jira/browse/SPARK-40858 ] Yikun Jiang deleted comment on SPARK-40858: - was (Author: apachespark): User 'Yikun' has created a pull request for this issue: https://github.com/apache/spark/pull/38408 > Cleanup github action warning > - > > Key: SPARK-40858 > URL: https://issues.apache.org/jira/browse/SPARK-40858 > Project: Spark > Issue Type: Bug > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > > * The `save-state` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * The `set-output` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * Node.js 12 actions are deprecated. For more information see: > https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. > Please update the following actions to use Node.js 16: actions/checkout, > docker/setup-qemu-action, docker/setup-buildx-action, > docker/build-push-action, docker/build-push-action, > docker/setup-buildx-action, actions/checkout > * > like: https://github.com/apache/spark/actions/runs/3288503290 > In order to reduce pontential impact on infra, we'd better address them one > by one according actions type. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40928) Upgrade actions/setup-python to v4
[ https://issues.apache.org/jira/browse/SPARK-40928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40928. - Resolution: Fixed > Upgrade actions/setup-python to v4 > -- > > Key: SPARK-40928 > URL: https://issues.apache.org/jira/browse/SPARK-40928 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40928) Upgrade actions/setup-python to v4
[ https://issues.apache.org/jira/browse/SPARK-40928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17624962#comment-17624962 ] Yikun Jiang commented on SPARK-40928: - Issue resolved by pull request 38408 https://github.com/apache/spark/pull/38408 > Upgrade actions/setup-python to v4 > -- > > Key: SPARK-40928 > URL: https://issues.apache.org/jira/browse/SPARK-40928 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40929) Add official image dockerfile for Spark v3.3.1
Yikun Jiang created SPARK-40929: --- Summary: Add official image dockerfile for Spark v3.3.1 Key: SPARK-40929 URL: https://issues.apache.org/jira/browse/SPARK-40929 Project: Spark Issue Type: Sub-task Components: Spark Docker Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40858) Cleanup github action warning
[ https://issues.apache.org/jira/browse/SPARK-40858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17624820#comment-17624820 ] Yikun Jiang commented on SPARK-40858: - [~hyukjin.kwon] According latest https://github.com/apache/spark/actions/runs/3326934877, only actions/setup-python left,. After https://issues.apache.org/jira/browse/SPARK-40928 all will be done. > Cleanup github action warning > - > > Key: SPARK-40858 > URL: https://issues.apache.org/jira/browse/SPARK-40858 > Project: Spark > Issue Type: Bug > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Priority: Major > > * The `save-state` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * The `set-output` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * Node.js 12 actions are deprecated. For more information see: > https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. > Please update the following actions to use Node.js 16: actions/checkout, > docker/setup-qemu-action, docker/setup-buildx-action, > docker/build-push-action, docker/build-push-action, > docker/setup-buildx-action, actions/checkout > * > like: https://github.com/apache/spark/actions/runs/3288503290 > In order to reduce pontential impact on infra, we'd better address them one > by one according actions type. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40928) Upgrade actions/setup-python to v4
Yikun Jiang created SPARK-40928: --- Summary: Upgrade actions/setup-python to v4 Key: SPARK-40928 URL: https://issues.apache.org/jira/browse/SPARK-40928 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40855) Add CONTRIBUTING.md to apache/spark-docker
[ https://issues.apache.org/jira/browse/SPARK-40855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40855. - Assignee: Qian Sun Resolution: Fixed > Add CONTRIBUTING.md to apache/spark-docker > -- > > Key: SPARK-40855 > URL: https://issues.apache.org/jira/browse/SPARK-40855 > Project: Spark > Issue Type: Sub-task > Components: Spark Docker >Affects Versions: 3.3.0 >Reporter: Qian Sun >Assignee: Qian Sun >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40855) Add CONTRIBUTING.md to apache/spark-docker
[ https://issues.apache.org/jira/browse/SPARK-40855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17623512#comment-17623512 ] Yikun Jiang commented on SPARK-40855: - Resolved by https://github.com/apache/spark-docker/pull/19 > Add CONTRIBUTING.md to apache/spark-docker > -- > > Key: SPARK-40855 > URL: https://issues.apache.org/jira/browse/SPARK-40855 > Project: Spark > Issue Type: Sub-task > Components: Spark Docker >Affects Versions: 3.3.0 >Reporter: Qian Sun >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-36057) SPIP: Support Customized Kubernetes Schedulers
[ https://issues.apache.org/jira/browse/SPARK-36057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17623509#comment-17623509 ] Yikun Jiang commented on SPARK-36057: - [~dongjoon] Thanks! > SPIP: Support Customized Kubernetes Schedulers > -- > > Key: SPARK-36057 > URL: https://issues.apache.org/jira/browse/SPARK-36057 > Project: Spark > Issue Type: Improvement > Components: Kubernetes >Affects Versions: 3.3.0 >Reporter: Holden Karau >Priority: Major > Labels: SPIP > Fix For: 3.3.1 > > > This is an umbrella issue for tracking the work for supporting Volcano & > Yunikorn on Kubernetes. These schedulers provide more YARN like features > (such as queues and minimum resources before scheduling jobs) that many folks > want on Kubernetes. > > Yunikorn is an ASF project & Volcano is a CNCF project (sig-batch). > > They've taken slightly different approaches to solving the same problem, but > from Spark's point of view we should be able to share much of the code. > > See the initial brainstorming discussion in SPARK-35623. > > DISCUSSION: [https://lists.apache.org/thread/zv3o62xrob4dvgkbftbv5w5wy75hkbxg] > VOTE: [https://lists.apache.org/thread/cz3cpp8q4pgmh7h35h6lvkwf6g3lwhcd] > VOTE Result: > [https://lists.apache.org/thread/nvwfo0yo0q8997vs86o7wkjyby4tbp0m] > Design DOC: > [https://docs.google.com/document/d/1xgQGRpaHQX6-QH_J9YV2C2Dh6RpXefUpLM7KGkzL6Fg] > Recap slide: > [https://lists.apache.org/thread/mwswfwkycj71npwz8gmv1r5nrvpwj77s] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40882) Upgrade actions/setup-java to v3 with distribution specified
[ https://issues.apache.org/jira/browse/SPARK-40882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40882: --- Assignee: Yikun Jiang > Upgrade actions/setup-java to v3 with distribution specified > > > Key: SPARK-40882 > URL: https://issues.apache.org/jira/browse/SPARK-40882 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40882) Upgrade actions/setup-java to v3 with distribution specified
[ https://issues.apache.org/jira/browse/SPARK-40882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40882. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38354 [https://github.com/apache/spark/pull/38354] > Upgrade actions/setup-java to v3 with distribution specified > > > Key: SPARK-40882 > URL: https://issues.apache.org/jira/browse/SPARK-40882 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40881) Upgrade actions/cache to v3 and actions/upload-artifact to v3
[ https://issues.apache.org/jira/browse/SPARK-40881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40881. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38353 [https://github.com/apache/spark/pull/38353] > Upgrade actions/cache to v3 and actions/upload-artifact to v3 > - > > Key: SPARK-40881 > URL: https://issues.apache.org/jira/browse/SPARK-40881 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40881) Upgrade actions/cache to v3 and actions/upload-artifact to v3
[ https://issues.apache.org/jira/browse/SPARK-40881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40881: --- Assignee: Yikun Jiang > Upgrade actions/cache to v3 and actions/upload-artifact to v3 > - > > Key: SPARK-40881 > URL: https://issues.apache.org/jira/browse/SPARK-40881 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-40881) Upgrade actions/cache to v3 and actions/upload-artifact to v3
[ https://issues.apache.org/jira/browse/SPARK-40881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40881: Summary: Upgrade actions/cache to v3 and actions/upload-artifact to v3 (was: Upgrade actions/cache and actions/upload-artifact) > Upgrade actions/cache to v3 and actions/upload-artifact to v3 > - > > Key: SPARK-40881 > URL: https://issues.apache.org/jira/browse/SPARK-40881 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40882) Upgrade actions/setup-java to v3 with distribution specified
Yikun Jiang created SPARK-40882: --- Summary: Upgrade actions/setup-java to v3 with distribution specified Key: SPARK-40882 URL: https://issues.apache.org/jira/browse/SPARK-40882 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40881) Upgrade actions/cache and actions/upload-artifact
Yikun Jiang created SPARK-40881: --- Summary: Upgrade actions/cache and actions/upload-artifact Key: SPARK-40881 URL: https://issues.apache.org/jira/browse/SPARK-40881 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-40871) Upgrade actions/script to v6 and fix notify workflow
[ https://issues.apache.org/jira/browse/SPARK-40871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40871: Summary: Upgrade actions/script to v6 and fix notify workflow (was: Upgrade actions/script to v6) > Upgrade actions/script to v6 and fix notify workflow > > > Key: SPARK-40871 > URL: https://issues.apache.org/jira/browse/SPARK-40871 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40870) Upgrade docker actions to cleanup warning
[ https://issues.apache.org/jira/browse/SPARK-40870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40870: --- Assignee: Yikun Jiang > Upgrade docker actions to cleanup warning > - > > Key: SPARK-40870 > URL: https://issues.apache.org/jira/browse/SPARK-40870 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > > docker/setup-qemu-action@v2 > docker/setup-buildx-action@v2 > docker/build-push-action@v3 > docker/login-action@v2 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40870) Upgrade docker actions to cleanup warning
[ https://issues.apache.org/jira/browse/SPARK-40870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40870. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38342 [https://github.com/apache/spark/pull/38342] > Upgrade docker actions to cleanup warning > - > > Key: SPARK-40870 > URL: https://issues.apache.org/jira/browse/SPARK-40870 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > > docker/setup-qemu-action@v2 > docker/setup-buildx-action@v2 > docker/build-push-action@v3 > docker/login-action@v2 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40870) Upgrade docker actions to cleanup warning
Yikun Jiang created SPARK-40870: --- Summary: Upgrade docker actions to cleanup warning Key: SPARK-40870 URL: https://issues.apache.org/jira/browse/SPARK-40870 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang docker/setup-qemu-action@v2 docker/setup-buildx-action@v2 docker/build-push-action@v3 docker/login-action@v2 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40871) Upgrade actions/script to v6
Yikun Jiang created SPARK-40871: --- Summary: Upgrade actions/script to v6 Key: SPARK-40871 URL: https://issues.apache.org/jira/browse/SPARK-40871 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40864) Remove pip/setuptools dynamic upgrade
[ https://issues.apache.org/jira/browse/SPARK-40864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40864. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 17 [https://github.com/apache/spark-docker/pull/17] > Remove pip/setuptools dynamic upgrade > - > > Key: SPARK-40864 > URL: https://issues.apache.org/jira/browse/SPARK-40864 > Project: Spark > Issue Type: Sub-task > Components: Spark Docker >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40864) Remove pip/setuptools dynamic upgrade
[ https://issues.apache.org/jira/browse/SPARK-40864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40864: --- Assignee: Yikun Jiang > Remove pip/setuptools dynamic upgrade > - > > Key: SPARK-40864 > URL: https://issues.apache.org/jira/browse/SPARK-40864 > Project: Spark > Issue Type: Sub-task > Components: Spark Docker >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40866) Rename Check Spark repo as Check Spark Docker repo in GA
[ https://issues.apache.org/jira/browse/SPARK-40866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40866: --- Assignee: Qian Sun > Rename Check Spark repo as Check Spark Docker repo in GA > > > Key: SPARK-40866 > URL: https://issues.apache.org/jira/browse/SPARK-40866 > Project: Spark > Issue Type: Sub-task > Components: Spark Docker >Affects Versions: 3.3.0 >Reporter: Qian Sun >Assignee: Qian Sun >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40866) Rename Check Spark repo as Check Spark Docker repo in GA
[ https://issues.apache.org/jira/browse/SPARK-40866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40866. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 18 [https://github.com/apache/spark-docker/pull/18] > Rename Check Spark repo as Check Spark Docker repo in GA > > > Key: SPARK-40866 > URL: https://issues.apache.org/jira/browse/SPARK-40866 > Project: Spark > Issue Type: Sub-task > Components: Spark Docker >Affects Versions: 3.3.0 >Reporter: Qian Sun >Assignee: Qian Sun >Priority: Minor > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40864) Remove pip/setuptools dynamic upgrade
Yikun Jiang created SPARK-40864: --- Summary: Remove pip/setuptools dynamic upgrade Key: SPARK-40864 URL: https://issues.apache.org/jira/browse/SPARK-40864 Project: Spark Issue Type: Sub-task Components: Spark Docker Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40860) Change `set-output` to `GITHUB_EVENT` in spark infra code
[ https://issues.apache.org/jira/browse/SPARK-40860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40860. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38323 [https://github.com/apache/spark/pull/38323] > Change `set-output` to `GITHUB_EVENT` in spark infra code > - > > Key: SPARK-40860 > URL: https://issues.apache.org/jira/browse/SPARK-40860 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40860) Change `set-output` to `GITHUB_EVENT` in spark infra code
[ https://issues.apache.org/jira/browse/SPARK-40860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40860: --- Assignee: Yikun Jiang > Change `set-output` to `GITHUB_EVENT` in spark infra code > - > > Key: SPARK-40860 > URL: https://issues.apache.org/jira/browse/SPARK-40860 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40859) Upgrade action/checkout to v3
[ https://issues.apache.org/jira/browse/SPARK-40859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40859. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38322 [https://github.com/apache/spark/pull/38322] > Upgrade action/checkout to v3 > - > > Key: SPARK-40859 > URL: https://issues.apache.org/jira/browse/SPARK-40859 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40859) Upgrade action/checkout to v3
[ https://issues.apache.org/jira/browse/SPARK-40859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40859: --- Assignee: Yikun Jiang > Upgrade action/checkout to v3 > - > > Key: SPARK-40859 > URL: https://issues.apache.org/jira/browse/SPARK-40859 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-40858) Cleanup github action warning
[ https://issues.apache.org/jira/browse/SPARK-40858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40858: Description: * The `save-state` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ * The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ * Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: actions/checkout, docker/setup-qemu-action, docker/setup-buildx-action, docker/build-push-action, docker/build-push-action, docker/setup-buildx-action, actions/checkout * like: https://github.com/apache/spark/actions/runs/3288503290 In order to reduce pontential impact on infra, we'd better address them one by one according actions type. was: * Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: actions/checkout, actions/checkout * The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ * Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: actions/checkout, docker/setup-qemu-action, docker/setup-buildx-action, docker/build-push-action, docker/build-push-action, docker/setup-buildx-action, actions/checkout * like: https://github.com/apache/spark/actions/runs/3288503290 In order to reduce pontential impact on infra, we'd better address them one by one according actions type. > Cleanup github action warning > - > > Key: SPARK-40858 > URL: https://issues.apache.org/jira/browse/SPARK-40858 > Project: Spark > Issue Type: Bug > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Priority: Major > > * The `save-state` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * The `set-output` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * Node.js 12 actions are deprecated. For more information see: > https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. > Please update the following actions to use Node.js 16: actions/checkout, > docker/setup-qemu-action, docker/setup-buildx-action, > docker/build-push-action, docker/build-push-action, > docker/setup-buildx-action, actions/checkout > * > like: https://github.com/apache/spark/actions/runs/3288503290 > In order to reduce pontential impact on infra, we'd better address them one > by one according actions type. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-40858) Cleanup github action warning
[ https://issues.apache.org/jira/browse/SPARK-40858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40858: Description: * Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: actions/checkout, actions/checkout * The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ * Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: actions/checkout, docker/setup-qemu-action, docker/setup-buildx-action, docker/build-push-action, docker/build-push-action, docker/setup-buildx-action, actions/checkout * like: https://github.com/apache/spark/actions/runs/3288503290 In order to reduce pontential impact on infra, we'd better address them one by one according actions type. was: * Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: actions/checkout, actions/checkout * The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ * Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: actions/checkout, docker/setup-qemu-action, docker/setup-buildx-action, docker/build-push-action, docker/build-push-action, docker/setup-buildx-action, actions/checkout In order to reduce pontential impact on infra, we'd better address them one by one according actions type. > Cleanup github action warning > - > > Key: SPARK-40858 > URL: https://issues.apache.org/jira/browse/SPARK-40858 > Project: Spark > Issue Type: Bug > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Priority: Major > > * Node.js 12 actions are deprecated. For more information see: > https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. > Please update the following actions to use Node.js 16: actions/checkout, > actions/checkout > * The `set-output` command is deprecated and will be disabled soon. Please > upgrade to using Environment Files. For more information see: > https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ > * Node.js 12 actions are deprecated. For more information see: > https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. > Please update the following actions to use Node.js 16: actions/checkout, > docker/setup-qemu-action, docker/setup-buildx-action, > docker/build-push-action, docker/build-push-action, > docker/setup-buildx-action, actions/checkout > * > like: https://github.com/apache/spark/actions/runs/3288503290 > In order to reduce pontential impact on infra, we'd better address them one > by one according actions type. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40860) Change `set-output` to `GITHUB_EVENT` in spark infra code
Yikun Jiang created SPARK-40860: --- Summary: Change `set-output` to `GITHUB_EVENT` in spark infra code Key: SPARK-40860 URL: https://issues.apache.org/jira/browse/SPARK-40860 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40859) Upgrade action/checkout to v3
Yikun Jiang created SPARK-40859: --- Summary: Upgrade action/checkout to v3 Key: SPARK-40859 URL: https://issues.apache.org/jira/browse/SPARK-40859 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40858) Cleanup github action warning
Yikun Jiang created SPARK-40858: --- Summary: Cleanup github action warning Key: SPARK-40858 URL: https://issues.apache.org/jira/browse/SPARK-40858 Project: Spark Issue Type: Bug Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang * Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: actions/checkout, actions/checkout * The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ * Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: actions/checkout, docker/setup-qemu-action, docker/setup-buildx-action, docker/build-push-action, docker/build-push-action, docker/setup-buildx-action, actions/checkout In order to reduce pontential impact on infra, we'd better address them one by one according actions type. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40838) Upgrade infra base image to focal-20220922
[ https://issues.apache.org/jira/browse/SPARK-40838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40838: --- Assignee: Yikun Jiang > Upgrade infra base image to focal-20220922 > -- > > Key: SPARK-40838 > URL: https://issues.apache.org/jira/browse/SPARK-40838 > Project: Spark > Issue Type: Bug > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40838) Upgrade infra base image to focal-20220922
[ https://issues.apache.org/jira/browse/SPARK-40838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40838. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38304 [https://github.com/apache/spark/pull/38304] > Upgrade infra base image to focal-20220922 > -- > > Key: SPARK-40838 > URL: https://issues.apache.org/jira/browse/SPARK-40838 > Project: Spark > Issue Type: Bug > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40845) Add template support for SPARK_GPG_KEY
[ https://issues.apache.org/jira/browse/SPARK-40845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40845. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 16 [https://github.com/apache/spark-docker/pull/16] > Add template support for SPARK_GPG_KEY > -- > > Key: SPARK-40845 > URL: https://issues.apache.org/jira/browse/SPARK-40845 > Project: Spark > Issue Type: Sub-task > Components: Spark Docker >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40845) Add template support for SPARK_GPG_KEY
[ https://issues.apache.org/jira/browse/SPARK-40845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40845: --- Assignee: Yikun Jiang > Add template support for SPARK_GPG_KEY > -- > > Key: SPARK-40845 > URL: https://issues.apache.org/jira/browse/SPARK-40845 > Project: Spark > Issue Type: Sub-task > Components: Spark Docker >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40845) Add template support for SPARK_GPG_KEY
Yikun Jiang created SPARK-40845: --- Summary: Add template support for SPARK_GPG_KEY Key: SPARK-40845 URL: https://issues.apache.org/jira/browse/SPARK-40845 Project: Spark Issue Type: Sub-task Components: Spark Docker Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40838) Upgrade infra base image to focal-20220922
Yikun Jiang created SPARK-40838: --- Summary: Upgrade infra base image to focal-20220922 Key: SPARK-40838 URL: https://issues.apache.org/jira/browse/SPARK-40838 Project: Spark Issue Type: Bug Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40833) Cleanup apt lists cache in Dockerfile
[ https://issues.apache.org/jira/browse/SPARK-40833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40833. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 14 [https://github.com/apache/spark-docker/pull/14] > Cleanup apt lists cache in Dockerfile > - > > Key: SPARK-40833 > URL: https://issues.apache.org/jira/browse/SPARK-40833 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40833) Cleanup apt lists cache in Dockerfile
[ https://issues.apache.org/jira/browse/SPARK-40833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40833: --- Assignee: Yikun Jiang > Cleanup apt lists cache in Dockerfile > - > > Key: SPARK-40833 > URL: https://issues.apache.org/jira/browse/SPARK-40833 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40832) Add README for spark-docker
[ https://issues.apache.org/jira/browse/SPARK-40832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40832. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 13 [https://github.com/apache/spark-docker/pull/13] > Add README for spark-docker > --- > > Key: SPARK-40832 > URL: https://issues.apache.org/jira/browse/SPARK-40832 > Project: Spark > Issue Type: Sub-task > Components: Documentation >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40832) Add README for spark-docker
[ https://issues.apache.org/jira/browse/SPARK-40832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40832: --- Assignee: Yikun Jiang > Add README for spark-docker > --- > > Key: SPARK-40832 > URL: https://issues.apache.org/jira/browse/SPARK-40832 > Project: Spark > Issue Type: Sub-task > Components: Documentation >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40833) Cleanup apt lists cache in Dockerfile
Yikun Jiang created SPARK-40833: --- Summary: Cleanup apt lists cache in Dockerfile Key: SPARK-40833 URL: https://issues.apache.org/jira/browse/SPARK-40833 Project: Spark Issue Type: Sub-task Components: Kubernetes Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40832) Add README for spark-docker
Yikun Jiang created SPARK-40832: --- Summary: Add README for spark-docker Key: SPARK-40832 URL: https://issues.apache.org/jira/browse/SPARK-40832 Project: Spark Issue Type: Sub-task Components: Documentation Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40528) Add dockerfile template
[
https://issues.apache.org/jira/browse/SPARK-40528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang resolved SPARK-40528.
-
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 12
[https://github.com/apache/spark-docker/pull/12]
> Add dockerfile template
> ---
>
> Key: SPARK-40528
> URL: https://issues.apache.org/jira/browse/SPARK-40528
> Project: Spark
> Issue Type: Sub-task
> Components: Project Infra
>Affects Versions: 3.4.0
>Reporter: Yikun Jiang
>Assignee: Yikun Jiang
>Priority: Major
> Fix For: 3.4.0
>
>
> Python addtion:
>
> {code:java}
> $ diff scala2.12-java11-r-ubuntu/Dockerfile
> scala2.12-java11-python3-r-ubuntu/Dockerfile
> 31a32,33
> > apt install -y python3 python3-pip && \
> > pip3 install --upgrade pip setuptools && \
> 33a36
> > mkdir /opt/spark/python && \
> 68a72,73
> > mv python/pyspark /opt/spark/python/pyspark/; \
> > mv python/lib /opt/spark/python/lib/; \ {code}
>
> R specific:
>
> {code:java}
> $ diff scala2.12-java11-python3-ubuntu/Dockerfile
> scala2.12-java11-python3-r-ubuntu/Dockerfile
> 33a34
> > apt install -y r-base r-base-dev && \
> 72a74
> > mv R /opt/spark/; \
> 78a81
> > ENV R_HOME /usr/lib/R {code}
>
> We'd better
> * add a base template and apply R/python3 on demand.
> * Support specify spark version, java version, scala version
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40805) Use `spark` username in official image
[ https://issues.apache.org/jira/browse/SPARK-40805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40805. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 11 [https://github.com/apache/spark-docker/pull/11] > Use `spark` username in official image > -- > > Key: SPARK-40805 > URL: https://issues.apache.org/jira/browse/SPARK-40805 > Project: Spark > Issue Type: Sub-task > Components: Build, Kubernetes >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40528) Add dockerfile template
[
https://issues.apache.org/jira/browse/SPARK-40528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang reassigned SPARK-40528:
---
Assignee: Yikun Jiang
> Add dockerfile template
> ---
>
> Key: SPARK-40528
> URL: https://issues.apache.org/jira/browse/SPARK-40528
> Project: Spark
> Issue Type: Sub-task
> Components: Project Infra
>Affects Versions: 3.4.0
>Reporter: Yikun Jiang
>Assignee: Yikun Jiang
>Priority: Major
>
> Python addtion:
>
> {code:java}
> $ diff scala2.12-java11-r-ubuntu/Dockerfile
> scala2.12-java11-python3-r-ubuntu/Dockerfile
> 31a32,33
> > apt install -y python3 python3-pip && \
> > pip3 install --upgrade pip setuptools && \
> 33a36
> > mkdir /opt/spark/python && \
> 68a72,73
> > mv python/pyspark /opt/spark/python/pyspark/; \
> > mv python/lib /opt/spark/python/lib/; \ {code}
>
> R specific:
>
> {code:java}
> $ diff scala2.12-java11-python3-ubuntu/Dockerfile
> scala2.12-java11-python3-r-ubuntu/Dockerfile
> 33a34
> > apt install -y r-base r-base-dev && \
> 72a74
> > mv R /opt/spark/; \
> 78a81
> > ENV R_HOME /usr/lib/R {code}
>
> We'd better
> * add a base template and apply R/python3 on demand.
> * Support specify spark version, java version, scala version
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40805) Use `spark` username in official image
[ https://issues.apache.org/jira/browse/SPARK-40805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40805: --- Assignee: Yikun Jiang > Use `spark` username in official image > -- > > Key: SPARK-40805 > URL: https://issues.apache.org/jira/browse/SPARK-40805 > Project: Spark > Issue Type: Sub-task > Components: Build, Kubernetes >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40570) Add doc for Docker Setup in standalone mode
[ https://issues.apache.org/jira/browse/SPARK-40570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40570: --- Assignee: Qian Sun > Add doc for Docker Setup in standalone mode > --- > > Key: SPARK-40570 > URL: https://issues.apache.org/jira/browse/SPARK-40570 > Project: Spark > Issue Type: Sub-task > Components: Documentation >Affects Versions: 3.4.0 >Reporter: Qian Sun >Assignee: Qian Sun >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40569) Expose port for spark standalone mode
[ https://issues.apache.org/jira/browse/SPARK-40569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40569: --- Assignee: Qian Sun > Expose port for spark standalone mode > - > > Key: SPARK-40569 > URL: https://issues.apache.org/jira/browse/SPARK-40569 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Qian Sun >Assignee: Qian Sun >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40569) Expose port for spark standalone mode
[ https://issues.apache.org/jira/browse/SPARK-40569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618193#comment-17618193 ] Yikun Jiang commented on SPARK-40569: - [~bjornjorgensen] It's a pretty good example! > Expose port for spark standalone mode > - > > Key: SPARK-40569 > URL: https://issues.apache.org/jira/browse/SPARK-40569 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Qian Sun >Assignee: Qian Sun >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40517) Add DOI manifest file for Spark Docker Official Image
[ https://issues.apache.org/jira/browse/SPARK-40517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40517: --- Assignee: Yikun Jiang > Add DOI manifest file for Spark Docker Official Image > - > > Key: SPARK-40517 > URL: https://issues.apache.org/jira/browse/SPARK-40517 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > > [https://github.com/docker-library/official-images/tree/master/library] > Example: [https://github.com/Yikun/official-images/pull/5] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40517) Add DOI manifest file for Spark Docker Official Image
[ https://issues.apache.org/jira/browse/SPARK-40517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618192#comment-17618192 ] Yikun Jiang commented on SPARK-40517: - https://github.com/docker-library/official-images/pull/13089 > Add DOI manifest file for Spark Docker Official Image > - > > Key: SPARK-40517 > URL: https://issues.apache.org/jira/browse/SPARK-40517 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Priority: Major > > [https://github.com/docker-library/official-images/tree/master/library] > Example: [https://github.com/Yikun/official-images/pull/5] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40805) Use `spark` username in official image
Yikun Jiang created SPARK-40805: --- Summary: Use `spark` username in official image Key: SPARK-40805 URL: https://issues.apache.org/jira/browse/SPARK-40805 Project: Spark Issue Type: Sub-task Components: Build, Kubernetes Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40783) Enable Spark on K8s integration test for official dockerfiles
[ https://issues.apache.org/jira/browse/SPARK-40783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40783. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 9 [https://github.com/apache/spark-docker/pull/9] > Enable Spark on K8s integration test for official dockerfiles > - > > Key: SPARK-40783 > URL: https://issues.apache.org/jira/browse/SPARK-40783 > Project: Spark > Issue Type: Sub-task > Components: Project Infra, Tests >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40783) Enable Spark on K8s integration test for official dockerfiles
[ https://issues.apache.org/jira/browse/SPARK-40783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40783: --- Assignee: Yikun Jiang > Enable Spark on K8s integration test for official dockerfiles > - > > Key: SPARK-40783 > URL: https://issues.apache.org/jira/browse/SPARK-40783 > Project: Spark > Issue Type: Sub-task > Components: Project Infra, Tests >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40754) Add LICENSE and NOTICE for apache/spark-docker
[ https://issues.apache.org/jira/browse/SPARK-40754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40754. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 6 [https://github.com/apache/spark-docker/pull/6] > Add LICENSE and NOTICE for apache/spark-docker > -- > > Key: SPARK-40754 > URL: https://issues.apache.org/jira/browse/SPARK-40754 > Project: Spark > Issue Type: Sub-task > Components: Documentation >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40754) Add LICENSE and NOTICE for apache/spark-docker
[ https://issues.apache.org/jira/browse/SPARK-40754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40754: --- Assignee: Yikun Jiang > Add LICENSE and NOTICE for apache/spark-docker > -- > > Key: SPARK-40754 > URL: https://issues.apache.org/jira/browse/SPARK-40754 > Project: Spark > Issue Type: Sub-task > Components: Documentation >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40783) Enable Spark on K8s integration test for official dockerfiles
Yikun Jiang created SPARK-40783: --- Summary: Enable Spark on K8s integration test for official dockerfiles Key: SPARK-40783 URL: https://issues.apache.org/jira/browse/SPARK-40783 Project: Spark Issue Type: Sub-task Components: Project Infra, Tests Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-40746) Make Dockerfile build workflow work in apache repo
[ https://issues.apache.org/jira/browse/SPARK-40746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-40746: Summary: Make Dockerfile build workflow work in apache repo (was: Switch workflow trigger event to `pull_request_target`) > Make Dockerfile build workflow work in apache repo > -- > > Key: SPARK-40746 > URL: https://issues.apache.org/jira/browse/SPARK-40746 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40746) Switch workflow trigger event to `pull_request_target`
[ https://issues.apache.org/jira/browse/SPARK-40746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40746. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 7 [https://github.com/apache/spark-docker/pull/7] > Switch workflow trigger event to `pull_request_target` > -- > > Key: SPARK-40746 > URL: https://issues.apache.org/jira/browse/SPARK-40746 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40746) Switch workflow trigger event to `pull_request_target`
[ https://issues.apache.org/jira/browse/SPARK-40746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40746: --- Assignee: Yikun Jiang > Switch workflow trigger event to `pull_request_target` > -- > > Key: SPARK-40746 > URL: https://issues.apache.org/jira/browse/SPARK-40746 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40757) Add PULL_REQUEST_TEMPLATE for spark-docker
[ https://issues.apache.org/jira/browse/SPARK-40757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40757. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 8 [https://github.com/apache/spark-docker/pull/8] > Add PULL_REQUEST_TEMPLATE for spark-docker > -- > > Key: SPARK-40757 > URL: https://issues.apache.org/jira/browse/SPARK-40757 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40757) Add PULL_REQUEST_TEMPLATE for spark-docker
[ https://issues.apache.org/jira/browse/SPARK-40757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40757: --- Assignee: Yikun Jiang > Add PULL_REQUEST_TEMPLATE for spark-docker > -- > > Key: SPARK-40757 > URL: https://issues.apache.org/jira/browse/SPARK-40757 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40757) Add PULL_REQUEST_TEMPLATE for spark-docker
Yikun Jiang created SPARK-40757: --- Summary: Add PULL_REQUEST_TEMPLATE for spark-docker Key: SPARK-40757 URL: https://issues.apache.org/jira/browse/SPARK-40757 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40754) Add LICENSE and NOTICE for apache/spark-docker
Yikun Jiang created SPARK-40754: --- Summary: Add LICENSE and NOTICE for apache/spark-docker Key: SPARK-40754 URL: https://issues.apache.org/jira/browse/SPARK-40754 Project: Spark Issue Type: Sub-task Components: Documentation Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-40746) Switch workflow trigger event to `pull_request_target`
Yikun Jiang created SPARK-40746: --- Summary: Switch workflow trigger event to `pull_request_target` Key: SPARK-40746 URL: https://issues.apache.org/jira/browse/SPARK-40746 Project: Spark Issue Type: Sub-task Components: Project Infra Affects Versions: 3.4.0 Reporter: Yikun Jiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40516) Add official image dockerfile for Spark v3.3.0
[ https://issues.apache.org/jira/browse/SPARK-40516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40516. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 2 [https://github.com/apache/spark-docker/pull/2] > Add official image dockerfile for Spark v3.3.0 > -- > > Key: SPARK-40516 > URL: https://issues.apache.org/jira/browse/SPARK-40516 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes, PySpark, SparkR >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > > Example: [https://github.com/Yikun/spark-docker/tree/master/3.3.0] > Test: > https://github.com/Yikun/spark-docker/blob/master/.github/workflows/build_3.3.0.yaml > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-40516) Add official image dockerfile for Spark v3.3.0
[ https://issues.apache.org/jira/browse/SPARK-40516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40516: --- Assignee: Yikun Jiang > Add official image dockerfile for Spark v3.3.0 > -- > > Key: SPARK-40516 > URL: https://issues.apache.org/jira/browse/SPARK-40516 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes, PySpark, SparkR >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > > Example: [https://github.com/Yikun/spark-docker/tree/master/3.3.0] > Test: > https://github.com/Yikun/spark-docker/blob/master/.github/workflows/build_3.3.0.yaml > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-40727) Add merge_spark_docker_pr.py to help merge commit
[ https://issues.apache.org/jira/browse/SPARK-40727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40727. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 1 [https://github.com/apache/spark-docker/pull/1] > Add merge_spark_docker_pr.py to help merge commit > - > > Key: SPARK-40727 > URL: https://issues.apache.org/jira/browse/SPARK-40727 > Project: Spark > Issue Type: Sub-task > Components: Project Infra >Affects Versions: 3.4.0 >Reporter: Yikun Jiang >Assignee: Yikun Jiang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
