[
https://issues.apache.org/jira/browse/SPARK-27889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855104#comment-16855104
]
Xiangrui Meng commented on SPARK-27889:
---
[~smilegator] I assigned this ticket to y
[
https://issues.apache.org/jira/browse/SPARK-27887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27887:
-
Assignee: Xiangrui Meng
> Check python version and print deprecation warning if version
[
https://issues.apache.org/jira/browse/SPARK-27889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27889:
-
Assignee: Xiao Li
> Make development scripts under dev/ support Python 3
>
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854833#comment-16854833
]
Xiangrui Meng commented on SPARK-27886:
---
Created a PR here:
https://github.com/py
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27885.
---
Resolution: Done
> Announce deprecation of Python 2 support
> --
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27886:
-
Assignee: Xiangrui Meng
> Add Apache Spark project to https://python3statement.org/
> -
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854737#comment-16854737
]
Xiangrui Meng commented on SPARK-27885:
---
Announced at:
* https://spark.apache.org/
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27885:
-
Assignee: Xiangrui Meng
> Announce deprecation of Python 2 support
> --
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852276#comment-16852276
]
Xiangrui Meng commented on SPARK-27886:
---
By "at the end of year", you mean year 20
Xiangrui Meng created SPARK-27889:
-
Summary: Make development scripts under dev/ support Python 3
Key: SPARK-27889
URL: https://issues.apache.org/jira/browse/SPARK-27889
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27885:
--
Description:
* Draft the message.
* Update Spark website and announce deprecation of Python 2
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852088#comment-16852088
]
Xiangrui Meng commented on SPARK-27886:
---
cc: [~srowen] [~smilegator]
> Add Apache
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27886:
--
Description:
Add Spark to https://python3statement.org/ and indicate our timeline. I
reviewed
[
https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27886:
--
Description:
Add Spark to https://python3statement.org/ and indicate our timeline. I
reviewed
[
https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852077#comment-16852077
]
Xiangrui Meng commented on SPARK-27884:
---
[~srowen] Could you help review the draft
[
https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852071#comment-16852071
]
Xiangrui Meng commented on SPARK-27884:
---
Draft message:
*Apache Spark's plan for
[
https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27884:
--
Description: Officially deprecate Python 2 support in Spark 3.0. (was:
Officially deprecate P
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27885:
--
Description:
* Draft the message.
* Update Spark website and announce deprecation of Python 2
Xiangrui Meng created SPARK-27888:
-
Summary: Python 2->3 migration guide for PySpark users
Key: SPARK-27888
URL: https://issues.apache.org/jira/browse/SPARK-27888
Project: Spark
Issue Type: S
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27885:
--
Summary: Announce deprecation of Python 2 support (was: Update Spark
website and put deprecat
[
https://issues.apache.org/jira/browse/SPARK-27887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27887:
--
Description: In Spark 3.0, users should see a deprecation warning if they
use PySpark with Pyt
Xiangrui Meng created SPARK-27887:
-
Summary: Check python version and print deprecation warning if
version < 3
Key: SPARK-27887
URL: https://issues.apache.org/jira/browse/SPARK-27887
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-27885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27885:
--
Description: Update Spark website and announce deprecation of Python 2
support in the next maj
Xiangrui Meng created SPARK-27886:
-
Summary: Add Apache Spark project to https://python3statement.org/
Key: SPARK-27886
URL: https://issues.apache.org/jira/browse/SPARK-27886
Project: Spark
I
Xiangrui Meng created SPARK-27885:
-
Summary: Update Spark website and put deprecation warning
Key: SPARK-27885
URL: https://issues.apache.org/jira/browse/SPARK-27885
Project: Spark
Issue Type
Xiangrui Meng created SPARK-27884:
-
Summary: Deprecate Python 2 support in Spark 3.0
Key: SPARK-27884
URL: https://issues.apache.org/jira/browse/SPARK-27884
Project: Spark
Issue Type: Story
Xiangrui Meng created SPARK-27823:
-
Summary: Add an abstraction layer for accelerator resource
handling to avoid manipulating raw confs
Key: SPARK-27823
URL: https://issues.apache.org/jira/browse/SPARK-27823
[
https://issues.apache.org/jira/browse/SPARK-27488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27488.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24615
[https://
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Description:
Pandas UDF is the ideal connection between PySpark and DL model inference
worklo
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Description:
Pandas UDF is the ideal connection between PySpark and DL model inference
worklo
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837353#comment-16837353
]
Xiangrui Meng commented on SPARK-26412:
---
[~WeichenXu123] I updated the description
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Summary: Allow Pandas UDF to take an iterator of pd.DataFrames (was: Allow
Pandas UDF to take
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-26412:
-
Assignee: Weichen Xu
> Allow Pandas UDF to take an iterator of pd.DataFrames or Arrow b
[
https://issues.apache.org/jira/browse/SPARK-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835780#comment-16835780
]
Xiangrui Meng commented on SPARK-27657:
---
This is how JDK format the error string:
[
https://issues.apache.org/jira/browse/SPARK-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835780#comment-16835780
]
Xiangrui Meng edited comment on SPARK-27657 at 5/8/19 5:42 PM:
---
[
https://issues.apache.org/jira/browse/SPARK-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835779#comment-16835779
]
Xiangrui Meng commented on SPARK-27657:
---
[~mrbago] Can you send a PR to fix it?
>
Xiangrui Meng created SPARK-27657:
-
Summary: ml.util.Instrumentation.logFailure doesn't log error
message
Key: SPARK-27657
URL: https://issues.apache.org/jira/browse/SPARK-27657
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-27588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27588.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24483
[https://
[
https://issues.apache.org/jira/browse/SPARK-27472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27472.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24484
[https://
[
https://issues.apache.org/jira/browse/SPARK-27472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27472:
-
Assignee: Xiangrui Meng
> Docuement binary file data source in Spark user guide
> -
[
https://issues.apache.org/jira/browse/SPARK-27588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27588:
-
Assignee: Xiangrui Meng
> Fail fast if binary file data source will load a file that is
Xiangrui Meng created SPARK-27588:
-
Summary: Fail fast if binary file data source will load a file
that is bigger than 2GB
Key: SPARK-27588
URL: https://issues.apache.org/jira/browse/SPARK-27588
Proje
[
https://issues.apache.org/jira/browse/SPARK-27534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27534.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24473
[https://
Xiangrui Meng created SPARK-27569:
-
Summary: Pandas UDF prefetches Arrow batches in the queue while
executing the current batch
Key: SPARK-27569
URL: https://issues.apache.org/jira/browse/SPARK-27569
[
https://issues.apache.org/jira/browse/SPARK-27312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27312:
-
Assignee: Weichen Xu
> PropertyGraph <-> GraphX conversions
> -
[
https://issues.apache.org/jira/browse/SPARK-27300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27300:
-
Assignee: Martin Junghanns (was: Xiangrui Meng)
> Create the new graph projects in Spa
[
https://issues.apache.org/jira/browse/SPARK-27300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27300:
-
Assignee: Xiangrui Meng
> Create the new graph projects in Spark and set up build/test
[
https://issues.apache.org/jira/browse/SPARK-27534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27534:
-
Assignee: Weichen Xu
> Do not load `content` column in binary data source if it is not
Xiangrui Meng created SPARK-27534:
-
Summary: Do not load `content` column in binary data source if it
is not selected
Key: SPARK-27534
URL: https://issues.apache.org/jira/browse/SPARK-27534
Project: S
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818640#comment-16818640
]
Xiangrui Meng edited comment on SPARK-25348 at 4/21/19 7:49 PM:
--
[
https://issues.apache.org/jira/browse/SPARK-27473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27473.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24387
[https://
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Summary: Allow Pandas UDF to take an iterator of pd.DataFrames or Arrow
batches for the entire
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Summary: Allow Pandas UDF to take an iterator of pd.DataFrames or Arrow
batches (was: Allow P
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822521#comment-16822521
]
Xiangrui Meng edited comment on SPARK-26412 at 4/20/19 5:13 PM:
--
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822521#comment-16822521
]
Xiangrui Meng edited comment on SPARK-26412 at 4/20/19 5:13 PM:
--
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-26412:
--
Description:
Pandas UDF is the ideal connection between PySpark and DL model inference
worklo
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822521#comment-16822521
]
Xiangrui Meng commented on SPARK-26412:
---
[~bryanc] It handles the data exchange fo
[
https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822517#comment-16822517
]
Xiangrui Meng edited comment on SPARK-27396 at 4/20/19 5:03 PM:
--
[
https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822517#comment-16822517
]
Xiangrui Meng edited comment on SPARK-27396 at 4/20/19 5:02 PM:
--
[
https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822517#comment-16822517
]
Xiangrui Meng edited comment on SPARK-27396 at 4/20/19 5:01 PM:
--
[
https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822517#comment-16822517
]
Xiangrui Meng commented on SPARK-27396:
---
[~revans2] Thanks for clarifying the prop
[
https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822367#comment-16822367
]
Xiangrui Meng commented on SPARK-27396:
---
[~revans2] What would end users do with p
[
https://issues.apache.org/jira/browse/SPARK-27473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16820500#comment-16820500
]
Xiangrui Meng commented on SPARK-27473:
---
Given SPARK-25558 is WIP, we might want t
[
https://issues.apache.org/jira/browse/SPARK-27473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27473:
-
Assignee: Weichen Xu
> Support filter push down for status fields in binary file data s
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-25348.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24354
[https://
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818640#comment-16818640
]
Xiangrui Meng commented on SPARK-25348:
---
I created two follow-up tasks:
* Document
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818640#comment-16818640
]
Xiangrui Meng edited comment on SPARK-25348 at 4/16/19 5:19 AM:
--
Xiangrui Meng created SPARK-27473:
-
Summary: Support filter push down for status fields in binary file
data source
Key: SPARK-27473
URL: https://issues.apache.org/jira/browse/SPARK-27473
Project: Spar
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25348:
--
Component/s: (was: ML)
> Data source for binary files
>
>
>
Xiangrui Meng created SPARK-27472:
-
Summary: Docuement binary file data source in Spark user guide
Key: SPARK-27472
URL: https://issues.apache.org/jira/browse/SPARK-27472
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25348:
--
Description:
It would be useful to have a data source implementation for binary files, which
[
https://issues.apache.org/jira/browse/SPARK-27454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-27454:
-
Assignee: Weichen Xu
> Spark image datasource fail when encounter some illegal images
>
[
https://issues.apache.org/jira/browse/SPARK-27454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-27454.
---
Resolution: Fixed
Fix Version/s: 3.0.0
> Spark image datasource fail when encounter s
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813834#comment-16813834
]
Xiangrui Meng commented on SPARK-25348:
---
Sampling could be supported later.
> Dat
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25348:
--
Description:
It would be useful to have a data source implementation for binary files, which
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813832#comment-16813832
]
Xiangrui Meng commented on SPARK-25348:
---
Updated the description and proposed APIs
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25348:
--
Description:
It would be useful to have a data source implementation for binary files, which
[
https://issues.apache.org/jira/browse/SPARK-25348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-25348:
-
Assignee: Weichen Xu
> Data source for binary files
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-27363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809511#comment-16809511
]
Xiangrui Meng commented on SPARK-27363:
---
[~felixcheung] [~srowen] Anyone you recom
[
https://issues.apache.org/jira/browse/SPARK-27363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27363:
--
Description: Design and implement Mesos support for GPU-aware scheduling.
> Mesos support for
[
https://issues.apache.org/jira/browse/SPARK-27362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27362:
--
Shepherd: Yinan Li
> Kubernetes support for GPU-aware scheduling
> ---
[
https://issues.apache.org/jira/browse/SPARK-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27365:
--
Description:
Upgrade Spark Jenkins to install GPU cards and run GPU integration tests
trigger
[
https://issues.apache.org/jira/browse/SPARK-27361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27361:
--
Description:
Design and implement YARN support for GPU-aware scheduling:
* User can request G
[
https://issues.apache.org/jira/browse/SPARK-27361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27361:
--
Description:
Design and implement YARN support for GPU-aware scheduling:
* User can request G
[
https://issues.apache.org/jira/browse/SPARK-27377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27377:
--
Description: This task should be covered by SPARK-23710. Just a placeholder
here.
> Upgrade Y
[
https://issues.apache.org/jira/browse/SPARK-27024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27024:
--
Issue Type: Story (was: Task)
> Executor interface for cluster managers to support GPU resour
[
https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-24615:
--
Epic Name: GPU-aware Scheduling (was: Support GPU Scheduling)
> SPIP: Accelerator-aware task
Xiangrui Meng created SPARK-27380:
-
Summary: Get and install GPU cards to Jenkins machines
Key: SPARK-27380
URL: https://issues.apache.org/jira/browse/SPARK-27380
Project: Spark
Issue Type: S
Xiangrui Meng created SPARK-27381:
-
Summary: Design: Spark Jenkins supports GPU integration tests
Key: SPARK-27381
URL: https://issues.apache.org/jira/browse/SPARK-27381
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27365:
--
Description: Upgrade Spark Jenkins to install GPU cards and run GPU
integration tests triggere
Xiangrui Meng created SPARK-27379:
-
Summary: YARN passes GPU info to Spark executor
Key: SPARK-27379
URL: https://issues.apache.org/jira/browse/SPARK-27379
Project: Spark
Issue Type: Sub-task
Xiangrui Meng created SPARK-27378:
-
Summary: spark-submit requests GPUs in YARN mode
Key: SPARK-27378
URL: https://issues.apache.org/jira/browse/SPARK-27378
Project: Spark
Issue Type: Sub-tas
Xiangrui Meng created SPARK-27377:
-
Summary: Upgrade YARN to 3.1.2+ to support GPU
Key: SPARK-27377
URL: https://issues.apache.org/jira/browse/SPARK-27377
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-27361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27361:
--
Description:
Design and implement YARN support for GPU-aware scheduling:
* User can request G
Xiangrui Meng created SPARK-27376:
-
Summary: Design: YARN supports Spark GPU-aware scheduling
Key: SPARK-27376
URL: https://issues.apache.org/jira/browse/SPARK-27376
Project: Spark
Issue Type
[
https://issues.apache.org/jira/browse/SPARK-27361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27361:
--
Description:
Design and implement YARN support for GPU-aware scheduling:
* User can request G
[
https://issues.apache.org/jira/browse/SPARK-27366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27366:
--
Description: Update Spark job scheduler to support accelerator resource
requests submitted at
Xiangrui Meng created SPARK-27374:
-
Summary: Fetch assigned resources from TaskContext
Key: SPARK-27374
URL: https://issues.apache.org/jira/browse/SPARK-27374
Project: Spark
Issue Type: Sub-t
[
https://issues.apache.org/jira/browse/SPARK-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27365:
--
Summary: Spark Jenkins supports testing GPU-aware scheduling features
(was: Spark Jenkins to
[
https://issues.apache.org/jira/browse/SPARK-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-27365:
--
Summary: Spark Jenkins to support testing GPU-aware scheduling features
(was: Spark Jenkins s
101 - 200 of 5507 matches
Mail list logo
