Bobby Wang created SPARK-47668:
--
Summary: Add API to remove the existing ResourceProfile from
ResourceProfileManager
Key: SPARK-47668
URL: https://issues.apache.org/jira/browse/SPARK-47668
Project:
Bobby Wang created SPARK-47663:
--
Summary: Add an end to end tests for checking if spark task works
well with resources
Key: SPARK-47663
URL: https://issues.apache.org/jira/browse/SPARK-47663
Project:
Bobby Wang created SPARK-47498:
--
Summary: Refine some fractional GPU resource calculation tests.
Key: SPARK-47498
URL: https://issues.apache.org/jira/browse/SPARK-47498
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-47458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bobby Wang updated SPARK-47458:
---
Summary: Incorrect to calculate the concurrent task number (was: Wrong to
calculate the concurrent
Bobby Wang created SPARK-47458:
--
Summary: Wrong to calculate the concurrent task number
Key: SPARK-47458
URL: https://issues.apache.org/jira/browse/SPARK-47458
Project: Spark
Issue Type: Bug
Bobby Wang created SPARK-47088:
--
Summary: Utilize BigDecimal to calculate the GPU resource
Key: SPARK-47088
URL: https://issues.apache.org/jira/browse/SPARK-47088
Project: Spark
Issue Type:
Bobby Wang created SPARK-46812:
--
Summary: Make `mapInPandas` / mapInArrow` support ResourceProfile
(Stage-Level scheduling)
Key: SPARK-46812
URL: https://issues.apache.org/jira/browse/SPARK-46812
[
https://issues.apache.org/jira/browse/SPARK-46715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17806754#comment-17806754
]
Bobby Wang commented on SPARK-46715:
Hi [~podongfeng], I run into the same issue for the PR
Bobby Wang created SPARK-46721:
--
Summary: Some tests failed when cherry-picking
https://github.com/apache/spark/pull/43494 back to branch 3.5
Key: SPARK-46721
URL: https://issues.apache.org/jira/browse/SPARK-46721
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795594#comment-17795594
]
Bobby Wang commented on SPARK-46032:
Thx [~haldefaiz] ,
I tried your patch with another two
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788743#comment-17788743
]
Bobby Wang commented on SPARK-46032:
Hi [~gurwls223], BTW, I tried it on the fourth machine this
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788717#comment-17788717
]
Bobby Wang commented on SPARK-46032:
Hi [~gurwls223], Any suggestions?
> connect: cannot assign
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788595#comment-17788595
]
Bobby Wang commented on SPARK-46032:
BTW, This morning, I tried it on the third machine, still got
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788584#comment-17788584
]
Bobby Wang commented on SPARK-46032:
Hi [~gurwls223], I've pasted all the logs, please tell me if
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788582#comment-17788582
]
Bobby Wang edited comment on SPARK-46032 at 11/22/23 12:54 AM:
---
h1.
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788583#comment-17788583
]
Bobby Wang commented on SPARK-46032:
Executor Logs,
{code:java}
Spark Executor Command:
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788582#comment-17788582
]
Bobby Wang commented on SPARK-46032:
h1. *Submit the spark application via pyspark*
I installed the
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788580#comment-17788580
]
Bobby Wang edited comment on SPARK-46032 at 11/22/23 12:46 AM:
---
h1.
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788581#comment-17788581
]
Bobby Wang commented on SPARK-46032:
h1. Start spark-connect-server which will start Spark Driver
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788580#comment-17788580
]
Bobby Wang commented on SPARK-46032:
h1. *Standalone Cluster*
h2. Master Log
{code:java}
Spark
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788384#comment-17788384
]
Bobby Wang commented on SPARK-46032:
I tried it on another machine and got the same issue. The scala
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788382#comment-17788382
]
Bobby Wang edited comment on SPARK-46032 at 11/21/23 11:52 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788382#comment-17788382
]
Bobby Wang edited comment on SPARK-46032 at 11/21/23 11:52 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788382#comment-17788382
]
Bobby Wang edited comment on SPARK-46032 at 11/21/23 11:51 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788382#comment-17788382
]
Bobby Wang commented on SPARK-46032:
{code:java}
~$ scala
Welcome to Scala 2.13.12 (OpenJDK 64-Bit
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788378#comment-17788378
]
Bobby Wang commented on SPARK-46032:
I tried it on another machine but still had the same problem.
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bobby Wang updated SPARK-46032:
---
Description:
I downloaded spark 3.5 from the spark official website, and then I started a
Spark
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bobby Wang updated SPARK-46032:
---
Description:
I downloaded spark 3.5 from the spark official website, and then I started a
Spark
[
https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bobby Wang updated SPARK-46032:
---
Description:
I downloaded spark 3.5 from the spark official website, and then I started a
Spark
Bobby Wang created SPARK-46032:
--
Summary: connect: cannot assign instance of
java.lang.invoke.SerializedLambda to field
org.apache.spark.rdd.MapPartitionsRDD.f
Key: SPARK-46032
URL:
[
https://issues.apache.org/jira/browse/SPARK-45527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775948#comment-17775948
]
Bobby Wang commented on SPARK-45527:
Working on a PR to fix it.
> Task fraction resource request is
Bobby Wang created SPARK-45495:
--
Summary: Support stage level task resource profile for k8s cluster
when dynamic allocation disabled
Key: SPARK-45495
URL: https://issues.apache.org/jira/browse/SPARK-45495
[
https://issues.apache.org/jira/browse/SPARK-45495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bobby Wang updated SPARK-45495:
---
Description:
[https://github.com/apache/spark/pull/37268] has introduced a new feature that
[
https://issues.apache.org/jira/browse/SPARK-45250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767527#comment-17767527
]
Bobby Wang commented on SPARK-45250:
Hi [~ivoson], May I ask why your previous PR
[
https://issues.apache.org/jira/browse/SPARK-45250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767522#comment-17767522
]
Bobby Wang commented on SPARK-45250:
I made a PR to fix it
Bobby Wang created SPARK-45250:
--
Summary: Support stage level task resource profile for yarn
cluster when dynamic allocation disabled
Key: SPARK-45250
URL: https://issues.apache.org/jira/browse/SPARK-45250
[
https://issues.apache.org/jira/browse/SPARK-40932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bobby Wang updated SPARK-40932:
---
Description:
When I was working on an internal project which has not been opened source. I
found
Bobby Wang created SPARK-40932:
--
Summary: Barrier: messages for allGather will be overridden by the
following barrier APIs
Key: SPARK-40932
URL: https://issues.apache.org/jira/browse/SPARK-40932
Bobby Wang created SPARK-40407:
--
Summary: Repartition of DataFrame can result in severe data skew
in some special case
Key: SPARK-40407
URL: https://issues.apache.org/jira/browse/SPARK-40407
Project:
[
https://issues.apache.org/jira/browse/SPARK-38911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522646#comment-17522646
]
Bobby Wang commented on SPARK-38911:
[~tgraves] Could you help to check that
> 'test 1 resource
[
https://issues.apache.org/jira/browse/SPARK-38911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522645#comment-17522645
]
Bobby Wang commented on SPARK-38911:
Just submited an PR for this issue
Bobby Wang created SPARK-38911:
--
Summary: 'test 1 resource profile' throws exception when running
it in IDEA separately
Key: SPARK-38911
URL: https://issues.apache.org/jira/browse/SPARK-38911
Project:
42 matches
Mail list logo
