[
https://issues.apache.org/jira/browse/SPARK-47978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BoYang resolved SPARK-47978.
Resolution: Fixed
> Decouple Spark Go Connect Library versioning from Spark versioning
>
BoYang created SPARK-47678:
--
Summary: Got fetch failed exception when new executor reused same
ip address from a previously killed executor
Key: SPARK-47678
URL: https://issues.apache.org/jira/browse/SPARK-47678
[
https://issues.apache.org/jira/browse/SPARK-44681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BoYang resolved SPARK-44681.
Fix Version/s: 3.4.0
Target Version/s: 3.5.0
Resolution: Fixed
> Solve issue referencing
BoYang created SPARK-44681:
--
Summary: Solve issue referencing
github.com/apache/spark-connect-go as Go library
Key: SPARK-44681
URL: https://issues.apache.org/jira/browse/SPARK-44681
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751186#comment-17751186
]
BoYang commented on SPARK-43351:
Thanks! We can keep it as 3.5.0 now.
> Support Golang in Spark Connect
BoYang created SPARK-44368:
--
Summary: Support partition operation on dataframe in Spark Connect
Go Client
Key: SPARK-44368
URL: https://issues.apache.org/jira/browse/SPARK-44368
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720241#comment-17720241
]
BoYang commented on SPARK-43351:
Yes, thanks Ruifeng for the comment and suggestion! Would like to add
BoYang created SPARK-43351:
--
Summary: Support Golang in Spark Connect
Key: SPARK-43351
URL: https://issues.apache.org/jira/browse/SPARK-43351
Project: Spark
Issue Type: New Feature
[
https://issues.apache.org/jira/browse/SPARK-38668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BoYang updated SPARK-38668:
---
Summary: Spark on Kubernetes: add separate pod watcher service to reduce
pressure on K8s API server (was:
[
https://issues.apache.org/jira/browse/SPARK-38668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BoYang updated SPARK-38668:
---
Description:
Spark driver will listen to all pods events to manage its executor pods. This
will cause
BoYang created SPARK-38668:
--
Summary: Spark on Kubernetes: support external pod watcher to
reduce pressure on K8s API server
Key: SPARK-38668
URL: https://issues.apache.org/jira/browse/SPARK-38668
Project:
[
https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457384#comment-17457384
]
BoYang edited comment on SPARK-25299 at 12/10/21, 10:01 PM:
I am working on
[
https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457384#comment-17457384
]
BoYang commented on SPARK-25299:
I am working on a prototype to store shuffle file on external storage
[
https://issues.apache.org/jira/browse/SPARK-34601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BoYang resolved SPARK-34601.
Resolution: Won't Fix
> Do not delete shuffle file on executor lost event when using remote shuffle
>
[
https://issues.apache.org/jira/browse/SPARK-34601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17296390#comment-17296390
]
BoYang commented on SPARK-34601:
It looks Spark already checks and matches the executor id when it tries
[
https://issues.apache.org/jira/browse/SPARK-34601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BoYang updated SPARK-34601:
---
Description: There are multiple work going on with disaggregated/remote
shuffle service (e.g. [LinkedIn
[
https://issues.apache.org/jira/browse/SPARK-34601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17294062#comment-17294062
]
BoYang commented on SPARK-34601:
I will add a PR for this soon.
> Do not delete shuffle file on
BoYang created SPARK-34601:
--
Summary: Do not delete shuffle file on executor lost event when
using remote shuffle service
Key: SPARK-34601
URL: https://issues.apache.org/jira/browse/SPARK-34601
Project:
BoYang created SPARK-33114:
--
Summary: Add metadata in MapStatus to support custom shuffle
manager
Key: SPARK-33114
URL: https://issues.apache.org/jira/browse/SPARK-33114
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17207093#comment-17207093
]
BoYang edited comment on SPARK-25299 at 10/4/20, 5:27 AM:
--
While people work on
[
https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17207093#comment-17207093
]
BoYang edited comment on SPARK-25299 at 10/4/20, 5:23 AM:
--
While people work on
[
https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17207093#comment-17207093
]
BoYang commented on SPARK-25299:
While people work on remote storage for persisting shuffle data, and
[
https://issues.apache.org/jira/browse/SPARK-33037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205743#comment-17205743
]
BoYang commented on SPARK-33037:
After discussion, we feel it is better to remove the knownManagers
[
https://issues.apache.org/jira/browse/SPARK-33037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BoYang updated SPARK-33037:
---
Summary: Remove knownManagers hardcoded list (was: Add
"spark.shuffle.manager" value to knownManagers)
>
[
https://issues.apache.org/jira/browse/SPARK-33037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BoYang updated SPARK-33037:
---
Description:
Spark has a hardcode list to contain known shuffle managers, which has two
values now. It
BoYang created SPARK-33037:
--
Summary: Add "spark.shuffle.manager" value to knownManagers
Key: SPARK-33037
URL: https://issues.apache.org/jira/browse/SPARK-33037
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-31924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159550#comment-17159550
]
BoYang edited comment on SPARK-31924 at 7/16/20, 11:21 PM:
---
We created a short
[
https://issues.apache.org/jira/browse/SPARK-31924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159550#comment-17159550
]
BoYang edited comment on SPARK-31924 at 7/16/20, 11:21 PM:
---
We created a short
[
https://issues.apache.org/jira/browse/SPARK-31924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159550#comment-17159550
]
BoYang commented on SPARK-31924:
We created a short [design
[
https://issues.apache.org/jira/browse/SPARK-31924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BoYang updated SPARK-31924:
---
Description:
People in [Spark Scalability & Reliability Sync Meeting
BoYang created SPARK-31924:
--
Summary: Create remote shuffle service reference implementation
Key: SPARK-31924
URL: https://issues.apache.org/jira/browse/SPARK-31924
Project: Spark
Issue Type: New
[
https://issues.apache.org/jira/browse/SPARK-29472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953209#comment-16953209
]
BoYang commented on SPARK-29472:
This is a pretty good feature, helping to solve production issue when
32 matches
Mail list logo
