[
https://issues.apache.org/jira/browse/SPARK-27889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855104#comment-16855104
]
Xiangrui Meng commented on SPARK-27889:
---
[~smilegator] I assigned this ticket to you for auditing
[
https://issues.apache.org/jira/browse/SPARK-27887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27887:
-
Assignee: Xiangrui Meng
> Check python version and print deprecation warning if
[
https://issues.apache.org/jira/browse/SPARK-27889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27889:
-
Assignee: Xiao Li
> Make development scripts under dev/ support Python 3
>
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854833#comment-16854833
]
Xiangrui Meng commented on SPARK-27886:
---
Created a PR here:
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27885.
---
Resolution: Done
> Announce deprecation of Python 2 support
>
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27886:
-
Assignee: Xiangrui Meng
> Add Apache Spark project to https://python3statement.org/
>
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854737#comment-16854737
]
Xiangrui Meng commented on SPARK-27885:
---
Announced at:
*
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27885:
-
Assignee: Xiangrui Meng
> Announce deprecation of Python 2 support
>
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852276#comment-16852276
]
Xiangrui Meng commented on SPARK-27886:
---
By "at the end of year", you mean year 2020, right? The
Xiangrui Meng created SPARK-27889:
-
Summary: Make development scripts under dev/ support Python 3
Key: SPARK-27889
URL: https://issues.apache.org/jira/browse/SPARK-27889
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27885:
--
Description:
* Draft the message.
* Update Spark website and announce deprecation of Python 2
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852088#comment-16852088
]
Xiangrui Meng commented on SPARK-27886:
---
cc: [~srowen] [~smilegator]
> Add Apache Spark project
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27886:
--
Description:
Add Spark to https://python3statement.org/ and indicate our timeline. I
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27886:
--
Description:
Add Spark to https://python3statement.org/ and indicate our timeline. I
[
https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852077#comment-16852077
]
Xiangrui Meng commented on SPARK-27884:
---
[~srowen] Could you help review the draft? Feel free to
[
https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852071#comment-16852071
]
Xiangrui Meng commented on SPARK-27884:
---
Draft message:
*Apache Spark's plan for dropping Python
[
https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27884:
--
Description: Officially deprecate Python 2 support in Spark 3.0. (was:
Officially deprecate
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27885:
--
Description:
* Draft the message.
* Update Spark website and announce deprecation of Python 2
Xiangrui Meng created SPARK-27888:
-
Summary: Python 2->3 migration guide for PySpark users
Key: SPARK-27888
URL: https://issues.apache.org/jira/browse/SPARK-27888
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27885:
--
Summary: Announce deprecation of Python 2 support (was: Update Spark
website and put
[
https://issues.apache.org/jira/browse/SPARK-27887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27887:
--
Description: In Spark 3.0, users should see a deprecation warning if they
use PySpark with
Xiangrui Meng created SPARK-27887:
-
Summary: Check python version and print deprecation warning if
version < 3
Key: SPARK-27887
URL: https://issues.apache.org/jira/browse/SPARK-27887
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27885:
--
Description: Update Spark website and announce deprecation of Python 2
support in the next
Xiangrui Meng created SPARK-27886:
-
Summary: Add Apache Spark project to https://python3statement.org/
Key: SPARK-27886
URL: https://issues.apache.org/jira/browse/SPARK-27886
Project: Spark
Xiangrui Meng created SPARK-27885:
-
Summary: Update Spark website and put deprecation warning
Key: SPARK-27885
URL: https://issues.apache.org/jira/browse/SPARK-27885
Project: Spark
Issue
Xiangrui Meng created SPARK-27884:
-
Summary: Deprecate Python 2 support in Spark 3.0
Key: SPARK-27884
URL: https://issues.apache.org/jira/browse/SPARK-27884
Project: Spark
Issue Type: Story
Xiangrui Meng created SPARK-27823:
-
Summary: Add an abstraction layer for accelerator resource
handling to avoid manipulating raw confs
Key: SPARK-27823
URL: https://issues.apache.org/jira/browse/SPARK-27823
[
https://issues.apache.org/jira/browse/SPARK-27488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27488.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24615
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Description:
Pandas UDF is the ideal connection between PySpark and DL model inference
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Description:
Pandas UDF is the ideal connection between PySpark and DL model inference
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837353#comment-16837353
]
Xiangrui Meng commented on SPARK-26412:
---
[~WeichenXu123] I updated the description.
> Allow
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Summary: Allow Pandas UDF to take an iterator of pd.DataFrames (was: Allow
Pandas UDF to
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-26412:
-
Assignee: Weichen Xu
> Allow Pandas UDF to take an iterator of pd.DataFrames or Arrow
[
https://issues.apache.org/jira/browse/SPARK-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835780#comment-16835780
]
Xiangrui Meng commented on SPARK-27657:
---
This is how JDK format the error string:
[
https://issues.apache.org/jira/browse/SPARK-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835780#comment-16835780
]
Xiangrui Meng edited comment on SPARK-27657 at 5/8/19 5:42 PM:
---
This is
[
https://issues.apache.org/jira/browse/SPARK-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835779#comment-16835779
]
Xiangrui Meng commented on SPARK-27657:
---
[~mrbago] Can you send a PR to fix it?
>
Xiangrui Meng created SPARK-27657:
-
Summary: ml.util.Instrumentation.logFailure doesn't log error
message
Key: SPARK-27657
URL: https://issues.apache.org/jira/browse/SPARK-27657
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-27588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27588.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24483
[
https://issues.apache.org/jira/browse/SPARK-27472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27472.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24484
[
https://issues.apache.org/jira/browse/SPARK-27472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27472:
-
Assignee: Xiangrui Meng
> Docuement binary file data source in Spark user guide
>
[
https://issues.apache.org/jira/browse/SPARK-27588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27588:
-
Assignee: Xiangrui Meng
> Fail fast if binary file data source will load a file that
Xiangrui Meng created SPARK-27588:
-
Summary: Fail fast if binary file data source will load a file
that is bigger than 2GB
Key: SPARK-27588
URL: https://issues.apache.org/jira/browse/SPARK-27588
[
https://issues.apache.org/jira/browse/SPARK-27534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27534.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24473
Xiangrui Meng created SPARK-27569:
-
Summary: Pandas UDF prefetches Arrow batches in the queue while
executing the current batch
Key: SPARK-27569
URL: https://issues.apache.org/jira/browse/SPARK-27569
[
https://issues.apache.org/jira/browse/SPARK-27312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27312:
-
Assignee: Weichen Xu
> PropertyGraph <-> GraphX conversions
>
[
https://issues.apache.org/jira/browse/SPARK-27300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27300:
-
Assignee: Martin Junghanns (was: Xiangrui Meng)
> Create the new graph projects in
[
https://issues.apache.org/jira/browse/SPARK-27300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27300:
-
Assignee: Xiangrui Meng
> Create the new graph projects in Spark and set up build/test
[
https://issues.apache.org/jira/browse/SPARK-27534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27534:
-
Assignee: Weichen Xu
> Do not load `content` column in binary data source if it is not
Xiangrui Meng created SPARK-27534:
-
Summary: Do not load `content` column in binary data source if it
is not selected
Key: SPARK-27534
URL: https://issues.apache.org/jira/browse/SPARK-27534
Project:
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818640#comment-16818640
]
Xiangrui Meng edited comment on SPARK-25348 at 4/21/19 7:49 PM:
I
[
https://issues.apache.org/jira/browse/SPARK-27473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27473.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24387
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Summary: Allow Pandas UDF to take an iterator of pd.DataFrames or Arrow
batches for the
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Summary: Allow Pandas UDF to take an iterator of pd.DataFrames or Arrow
batches (was: Allow
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822521#comment-16822521
]
Xiangrui Meng edited comment on SPARK-26412 at 4/20/19 5:13 PM:
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822521#comment-16822521
]
Xiangrui Meng edited comment on SPARK-26412 at 4/20/19 5:13 PM:
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Description:
Pandas UDF is the ideal connection between PySpark and DL model inference
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822521#comment-16822521
]
Xiangrui Meng commented on SPARK-26412:
---
[~bryanc] It handles the data exchange for DL model
[
https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822517#comment-16822517
]
Xiangrui Meng edited comment on SPARK-27396 at 4/20/19 5:03 PM:
[
https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822517#comment-16822517
]
Xiangrui Meng edited comment on SPARK-27396 at 4/20/19 5:02 PM:
[
https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822517#comment-16822517
]
Xiangrui Meng edited comment on SPARK-27396 at 4/20/19 5:01 PM:
[
https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822517#comment-16822517
]
Xiangrui Meng commented on SPARK-27396:
---
[~revans2] Thanks for clarifying the proposal! If your
[
https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822367#comment-16822367
]
Xiangrui Meng commented on SPARK-27396:
---
[~revans2] What would end users do with public APIs for
[
https://issues.apache.org/jira/browse/SPARK-27473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820500#comment-16820500
]
Xiangrui Meng commented on SPARK-27473:
---
Given SPARK-25558 is WIP, we might want to flatten the
[
https://issues.apache.org/jira/browse/SPARK-27473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27473:
-
Assignee: Weichen Xu
> Support filter push down for status fields in binary file data
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-25348.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24354
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818640#comment-16818640
]
Xiangrui Meng commented on SPARK-25348:
---
I created two follow-up tasks:
* DocumentationL
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818640#comment-16818640
]
Xiangrui Meng edited comment on SPARK-25348 at 4/16/19 5:19 AM:
I
Xiangrui Meng created SPARK-27473:
-
Summary: Support filter push down for status fields in binary file
data source
Key: SPARK-27473
URL: https://issues.apache.org/jira/browse/SPARK-27473
Project:
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25348:
--
Component/s: (was: ML)
> Data source for binary files
>
>
>
Xiangrui Meng created SPARK-27472:
-
Summary: Docuement binary file data source in Spark user guide
Key: SPARK-27472
URL: https://issues.apache.org/jira/browse/SPARK-27472
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25348:
--
Description:
It would be useful to have a data source implementation for binary files, which
[
https://issues.apache.org/jira/browse/SPARK-27454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27454:
-
Assignee: Weichen Xu
> Spark image datasource fail when encounter some illegal images
[
https://issues.apache.org/jira/browse/SPARK-27454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27454.
---
Resolution: Fixed
Fix Version/s: 3.0.0
> Spark image datasource fail when encounter
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813834#comment-16813834
]
Xiangrui Meng commented on SPARK-25348:
---
Sampling could be supported later.
> Data source for
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25348:
--
Description:
It would be useful to have a data source implementation for binary files, which
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813832#comment-16813832
]
Xiangrui Meng commented on SPARK-25348:
---
Updated the description and proposed APIs.
> Data source
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25348:
--
Description:
It would be useful to have a data source implementation for binary files, which
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-25348:
-
Assignee: Weichen Xu
> Data source for binary files
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-27363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809511#comment-16809511
]
Xiangrui Meng commented on SPARK-27363:
---
[~felixcheung] [~srowen] Anyone you recommend to lead
[
https://issues.apache.org/jira/browse/SPARK-27363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27363:
--
Description: Design and implement Mesos support for GPU-aware scheduling.
> Mesos support for
[
https://issues.apache.org/jira/browse/SPARK-27362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27362:
--
Shepherd: Yinan Li
> Kubernetes support for GPU-aware scheduling
>
[
https://issues.apache.org/jira/browse/SPARK-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27365:
--
Description:
Upgrade Spark Jenkins to install GPU cards and run GPU integration tests
[
https://issues.apache.org/jira/browse/SPARK-27361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27361:
--
Description:
Design and implement YARN support for GPU-aware scheduling:
* User can request
[
https://issues.apache.org/jira/browse/SPARK-27361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27361:
--
Description:
Design and implement YARN support for GPU-aware scheduling:
* User can request
[
https://issues.apache.org/jira/browse/SPARK-27377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27377:
--
Description: This task should be covered by SPARK-23710. Just a placeholder
here.
> Upgrade
[
https://issues.apache.org/jira/browse/SPARK-27024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27024:
--
Issue Type: Story (was: Task)
> Executor interface for cluster managers to support GPU
[
https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-24615:
--
Epic Name: GPU-aware Scheduling (was: Support GPU Scheduling)
> SPIP: Accelerator-aware task
Xiangrui Meng created SPARK-27380:
-
Summary: Get and install GPU cards to Jenkins machines
Key: SPARK-27380
URL: https://issues.apache.org/jira/browse/SPARK-27380
Project: Spark
Issue Type:
Xiangrui Meng created SPARK-27381:
-
Summary: Design: Spark Jenkins supports GPU integration tests
Key: SPARK-27381
URL: https://issues.apache.org/jira/browse/SPARK-27381
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27365:
--
Description: Upgrade Spark Jenkins to install GPU cards and run GPU
integration tests
Xiangrui Meng created SPARK-27379:
-
Summary: YARN passes GPU info to Spark executor
Key: SPARK-27379
URL: https://issues.apache.org/jira/browse/SPARK-27379
Project: Spark
Issue Type:
Xiangrui Meng created SPARK-27378:
-
Summary: spark-submit requests GPUs in YARN mode
Key: SPARK-27378
URL: https://issues.apache.org/jira/browse/SPARK-27378
Project: Spark
Issue Type:
Xiangrui Meng created SPARK-27377:
-
Summary: Upgrade YARN to 3.1.2+ to support GPU
Key: SPARK-27377
URL: https://issues.apache.org/jira/browse/SPARK-27377
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-27361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27361:
--
Description:
Design and implement YARN support for GPU-aware scheduling:
* User can request
Xiangrui Meng created SPARK-27376:
-
Summary: Design: YARN supports Spark GPU-aware scheduling
Key: SPARK-27376
URL: https://issues.apache.org/jira/browse/SPARK-27376
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-27361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27361:
--
Description:
Design and implement YARN support for GPU-aware scheduling:
* User can request
[
https://issues.apache.org/jira/browse/SPARK-27366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27366:
--
Description: Update Spark job scheduler to support accelerator resource
requests submitted at
Xiangrui Meng created SPARK-27374:
-
Summary: Fetch assigned resources from TaskContext
Key: SPARK-27374
URL: https://issues.apache.org/jira/browse/SPARK-27374
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27365:
--
Summary: Spark Jenkins supports testing GPU-aware scheduling features
(was: Spark Jenkins to
[
https://issues.apache.org/jira/browse/SPARK-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27365:
--
Summary: Spark Jenkins to support testing GPU-aware scheduling features
(was: Spark Jenkins
101 - 200 of 5214 matches
Mail list logo
