[jira] [Updated] (SPARK-24037) stateful operators

[ https://issues.apache.org/jira/browse/SPARK-24037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24037: Fix Version/s: (was: 2.4.0) > stateful operators > -- > >

[jira] [Updated] (SPARK-24037) stateful operators

[ https://issues.apache.org/jira/browse/SPARK-24037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24037: Fix Version/s: 2.4.0 > stateful operators > -- > > Key:

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549276#comment-16549276 ] Saisai Shao commented on SPARK-24615: - Thanks [~tgraves] for the suggestion.  {quote}Once I get to

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548662#comment-16548662 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] I'm rewriting the design doc based on the

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547263#comment-16547263 ] Saisai Shao commented on SPARK-24615: - Sure, I will also add it as Xiangrui also suggested the same

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546590#comment-16546590 ] Saisai Shao commented on SPARK-24615: - Yes, currently the user is responsible for asking yarn to get

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545962#comment-16545962 ] Saisai Shao commented on SPARK-24615: - Sorry [~tgraves] for the late response. Yes,  when requesting

[jira] [Comment Edited] (SPARK-24804) There are duplicate words in the title in the DatasetSuite

[ https://issues.apache.org/jira/browse/SPARK-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544386#comment-16544386 ] Saisai Shao edited comment on SPARK-24804 at 7/15/18 1:29 AM: -- Please don't

[jira] [Commented] (SPARK-24804) There are duplicate words in the title in the DatasetSuite

[ https://issues.apache.org/jira/browse/SPARK-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544386#comment-16544386 ] Saisai Shao commented on SPARK-24804: - Please don't set target version or fix version. Committers

[jira] [Updated] (SPARK-24804) There are duplicate words in the title in the DatasetSuite

[ https://issues.apache.org/jira/browse/SPARK-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24804: Priority: Trivial (was: Minor) > There are duplicate words in the title in the DatasetSuite >

[jira] [Updated] (SPARK-24804) There are duplicate words in the title in the DatasetSuite

[ https://issues.apache.org/jira/browse/SPARK-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24804: Target Version/s: (was: 2.3.2) > There are duplicate words in the title in the DatasetSuite >

[jira] [Updated] (SPARK-24804) There are duplicate words in the title in the DatasetSuite

[ https://issues.apache.org/jira/browse/SPARK-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24804: Fix Version/s: (was: 2.3.2) > There are duplicate words in the title in the DatasetSuite >

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539635#comment-16539635 ] Saisai Shao commented on SPARK-24781: - I see. I will wait for this before cutting a new 2.3.2 RC

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539630#comment-16539630 ] Saisai Shao commented on SPARK-24781: - Thanks Felix. Does this have to be in 2.3.2? [~ueshin] >

[jira] [Assigned] (SPARK-24678) We should use 'PROCESS_LOCAL' first for Spark-Streaming

[ https://issues.apache.org/jira/browse/SPARK-24678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24678: --- Assignee: sharkd tu > We should use 'PROCESS_LOCAL' first for Spark-Streaming >

[jira] [Resolved] (SPARK-24678) We should use 'PROCESS_LOCAL' first for Spark-Streaming

[ https://issues.apache.org/jira/browse/SPARK-24678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24678. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21658

[jira] [Resolved] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24646. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21633

[jira] [Assigned] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24646: --- Assignee: Saisai Shao > Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes >

[jira] [Comment Edited] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534565#comment-16534565 ] Saisai Shao edited comment on SPARK-24723 at 7/6/18 8:25 AM: - [~mengxr]

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534565#comment-16534565 ] Saisai Shao commented on SPARK-24723: - [~mengxr] [~jiangxb1987] There's one solution to handle

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534529#comment-16534529 ] Saisai Shao commented on SPARK-24723: - After discussed with Xiangrui offline, resource reservation

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533205#comment-16533205 ] Saisai Shao commented on SPARK-24723: - Hi [~mengxr], I would like to know the goal of this ticket?

[jira] [Commented] (SPARK-24739) PySpark does not work with Python 3.7.0

[ https://issues.apache.org/jira/browse/SPARK-24739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533189#comment-16533189 ] Saisai Shao commented on SPARK-24739: - Do we have to fix it in 2.3.2? I don't think it is even

[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR on Windows

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532145#comment-16532145 ] Saisai Shao commented on SPARK-24535: - Hi [~felixcheung] what's the current status of this JIRA, do

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532143#comment-16532143 ] Saisai Shao commented on SPARK-24530: - Hi [~hyukjin.kwon] what is the current status of this JIRA,

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529352#comment-16529352 ] Saisai Shao commented on SPARK-24530: - [~hyukjin.kwon], Spark 2.1.3 and 2.2.2 are on vote, can you

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24530: Target Version/s: 2.1.3, 2.2.2, 2.3.2, 2.4.0 (was: 2.4.0) > Sphinx doesn't render

[jira] [Resolved] (SPARK-24418) Upgrade to Scala 2.11.12

[ https://issues.apache.org/jira/browse/SPARK-24418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24418. - Resolution: Fixed Issue resolved by pull request 21495

[jira] [Commented] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523022#comment-16523022 ] Saisai Shao commented on SPARK-24646: - Hi [~vanzin], here is a specific example: We have a

[jira] [Created] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

Saisai Shao created SPARK-24646: --- Summary: Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes Key: SPARK-24646 URL: https://issues.apache.org/jira/browse/SPARK-24646 Project: Spark

[jira] [Commented] (SPARK-24374) SPIP: Support Barrier Scheduling in Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-24374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519934#comment-16519934 ] Saisai Shao commented on SPARK-24374: - Hi [~mengxr] [~jiangxb1987] SPARK-24615 would be a part of

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24615: Labels: SPIP (was: ) > Accelerator aware task scheduling for Spark >

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24615: Description: In the machine learning area, accelerator card (GPU, FPGA, TPU) is predominant

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24615: Description: In the machine learning area, accelerator card (GPU, FPGA, TPU) is predominant

[jira] [Created] (SPARK-24615) Accelerator aware task scheduling for Spark

Saisai Shao created SPARK-24615: --- Summary: Accelerator aware task scheduling for Spark Key: SPARK-24615 URL: https://issues.apache.org/jira/browse/SPARK-24615 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517720#comment-16517720 ] Saisai Shao commented on SPARK-24493: - This BUG is fixed in HDFS-12670, Spark should bump its

[jira] [Updated] (SPARK-24518) Using Hadoop credential provider API to store password

[ https://issues.apache.org/jira/browse/SPARK-24518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24518: Description: Current Spark configs password in a plaintext way, like putting in the

[jira] [Updated] (SPARK-24518) Using Hadoop credential provider API to store password

[ https://issues.apache.org/jira/browse/SPARK-24518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24518: Environment: (was: Current Spark configs password in a plaintext way, like putting in the

[jira] [Created] (SPARK-24518) Using Hadoop credential provider API to store password

Saisai Shao created SPARK-24518: --- Summary: Using Hadoop credential provider API to store password Key: SPARK-24518 URL: https://issues.apache.org/jira/browse/SPARK-24518 Project: Spark Issue

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506023#comment-16506023 ] Saisai Shao commented on SPARK-23534: - Yes, it is a Hadoop 2.8+ issue, but we don't a 2.8 profile

[jira] [Resolved] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24487. - Resolution: Won't Fix > Add support for RabbitMQ. > - > >

[jira] [Commented] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505786#comment-16505786 ] Saisai Shao commented on SPARK-24487: - I'm sure Spark community will not merge this into Spark code

[jira] [Updated] (SPARK-23151) Provide a distribution of Spark with Hadoop 3.0

[ https://issues.apache.org/jira/browse/SPARK-23151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23151: Issue Type: Sub-task (was: Dependency upgrade) Parent: SPARK-23534 > Provide a

[jira] [Commented] (SPARK-23151) Provide a distribution of Spark with Hadoop 3.0

[ https://issues.apache.org/jira/browse/SPARK-23151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505783#comment-16505783 ] Saisai Shao commented on SPARK-23151: - I will convert this as a subtask of SPARK-23534. > Provide a

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505771#comment-16505771 ] Saisai Shao commented on SPARK-23534: - Spark with Hadoop 3 will be failed in token renew for long

[jira] [Comment Edited] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505768#comment-16505768 ] Saisai Shao edited comment on SPARK-24493 at 6/8/18 6:56 AM: - Adding more

[jira] [Commented] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505768#comment-16505768 ] Saisai Shao commented on SPARK-24493: - Adding more background, the issue is happened when building

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Summary: Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3 (was: Kerberos Ticket

[jira] [Commented] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505760#comment-16505760 ] Saisai Shao commented on SPARK-24487: - You can add it in your own package with DataSource API.

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in long running Spark job

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Component/s: Spark Core > Kerberos Ticket Renewal is failing in long running Spark job >

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in long running Spark job

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Component/s: (was: Spark Core) YARN > Kerberos Ticket Renewal is failing in

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in long running Spark job

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Priority: Major (was: Blocker) > Kerberos Ticket Renewal is failing in long running Spark job >

[jira] [Commented] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505608#comment-16505608 ] Saisai Shao commented on SPARK-24487: - What's the usage and purpose to integrate RabbitMQ to Spark?

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501225#comment-16501225 ] Saisai Shao commented on SPARK-20202: - OK, for the 1st, I've already started working on it locally.

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499707#comment-16499707 ] Saisai Shao commented on SPARK-20202: - What is our plan to to fix this issue, are we going to use

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497931#comment-16497931 ] Saisai Shao commented on SPARK-18673: - Thanks Steve, looking forward to your inputs. > Dataframes

[jira] [Commented] (SPARK-24355) Improve Spark shuffle server responsiveness to non-ChunkFetch requests

[ https://issues.apache.org/jira/browse/SPARK-24355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497626#comment-16497626 ] Saisai Shao commented on SPARK-24355: - Do we have a test result before and after? > Improve Spark

[jira] [Commented] (SPARK-24448) File not found on the address SparkFiles.get returns on standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-24448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497548#comment-16497548 ] Saisai Shao commented on SPARK-24448: - Does it only happen in standalone cluster mode, have you

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16496187#comment-16496187 ] Saisai Shao commented on SPARK-18673: - I created a PR for this issue

[jira] [Resolved] (SPARK-23991) data loss when allocateBlocksToBatch

[ https://issues.apache.org/jira/browse/SPARK-23991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23991. - Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by pull

[jira] [Assigned] (SPARK-23991) data loss when allocateBlocksToBatch

[ https://issues.apache.org/jira/browse/SPARK-23991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23991: --- Assignee: Gabor Somogyi > data loss when allocateBlocksToBatch >

[jira] [Created] (SPARK-24377) Make --py-files work in non pyspark application

Saisai Shao created SPARK-24377: --- Summary: Make --py-files work in non pyspark application Key: SPARK-24377 URL: https://issues.apache.org/jira/browse/SPARK-24377 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475480#comment-16475480 ] Saisai Shao commented on SPARK-24241: - Issue resolved by pull request 21290

[jira] [Resolved] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24241. - Resolution: Fixed > Do not fail fast when dynamic resource allocation enabled with 0 executor >

[jira] [Updated] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24241: Fix Version/s: 2.4.0 > Do not fail fast when dynamic resource allocation enabled with 0 executor >

[jira] [Assigned] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24241: --- Assignee: Kent Yao > Do not fail fast when dynamic resource allocation enabled with 0

[jira] [Assigned] (SPARK-24182) Improve error message for client mode when AM fails

[ https://issues.apache.org/jira/browse/SPARK-24182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24182: --- Assignee: Marcelo Vanzin > Improve error message for client mode when AM fails >

[jira] [Resolved] (SPARK-24182) Improve error message for client mode when AM fails

[ https://issues.apache.org/jira/browse/SPARK-24182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24182. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21243

[jira] [Created] (SPARK-24219) Improve the docker build script to avoid copying everything in example

Saisai Shao created SPARK-24219: --- Summary: Improve the docker build script to avoid copying everything in example Key: SPARK-24219 URL: https://issues.apache.org/jira/browse/SPARK-24219 Project: Spark

[jira] [Resolved] (SPARK-24188) /api/v1/version not working

[ https://issues.apache.org/jira/browse/SPARK-24188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24188. - Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by pull

[jira] [Assigned] (SPARK-24188) /api/v1/version not working

[ https://issues.apache.org/jira/browse/SPARK-24188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24188: --- Assignee: Marcelo Vanzin > /api/v1/version not working > --- > >

[jira] [Commented] (SPARK-24174) Expose Hadoop config as part of /environment API

[ https://issues.apache.org/jira/browse/SPARK-24174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463575#comment-16463575 ] Saisai Shao commented on SPARK-24174: - I believe Hadoop web UI already expose such configurations. It

[jira] [Assigned] (SPARK-24136) MemoryStreamDataReader.next should skip sleeping if record is available

[ https://issues.apache.org/jira/browse/SPARK-24136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24136: --- Assignee: Arun Mahadevan > MemoryStreamDataReader.next should skip sleeping if record is

[jira] [Resolved] (SPARK-24136) MemoryStreamDataReader.next should skip sleeping if record is available

[ https://issues.apache.org/jira/browse/SPARK-24136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24136. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21207

[jira] [Resolved] (SPARK-24110) Avoid calling UGI loginUserFromKeytab in ThriftServer

[ https://issues.apache.org/jira/browse/SPARK-24110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24110. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21178

[jira] [Assigned] (SPARK-24110) Avoid calling UGI loginUserFromKeytab in ThriftServer

[ https://issues.apache.org/jira/browse/SPARK-24110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24110: --- Assignee: Saisai Shao > Avoid calling UGI loginUserFromKeytab in ThriftServer >

[jira] [Assigned] (SPARK-23688) Refactor tests away from rate source

[ https://issues.apache.org/jira/browse/SPARK-23688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23688: --- Assignee: Jungtaek Lim > Refactor tests away from rate source >

[jira] [Resolved] (SPARK-23688) Refactor tests away from rate source

[ https://issues.apache.org/jira/browse/SPARK-23688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23688. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21152

[jira] [Assigned] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23830: --- Assignee: Eric Maynard > Spark on YARN in cluster deploy mode fail with

[jira] [Resolved] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23830. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21168

[jira] [Created] (SPARK-24110) Avoid calling UGI loginUserFromKeytab in ThriftServer

Saisai Shao created SPARK-24110: --- Summary: Avoid calling UGI loginUserFromKeytab in ThriftServer Key: SPARK-24110 URL: https://issues.apache.org/jira/browse/SPARK-24110 Project: Spark Issue

[jira] [Commented] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455621#comment-16455621 ] Saisai Shao commented on SPARK-23830: - I agree [~emaynard]. > Spark on YARN in cluster deploy mode

[jira] [Commented] (SPARK-24062) SASL encryption cannot be worked in ThriftServer

[ https://issues.apache.org/jira/browse/SPARK-24062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453521#comment-16453521 ] Saisai Shao commented on SPARK-24062: - Issue resolved by pull request 21138

[jira] [Resolved] (SPARK-24062) SASL encryption cannot be worked in ThriftServer

[ https://issues.apache.org/jira/browse/SPARK-24062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24062. - Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.4.0

[jira] [Created] (SPARK-24062) SASL encryption cannot be worked in ThriftServer

Saisai Shao created SPARK-24062: --- Summary: SASL encryption cannot be worked in ThriftServer Key: SPARK-24062 URL: https://issues.apache.org/jira/browse/SPARK-24062 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-22372) Make YARN client extend SparkApplication

[ https://issues.apache.org/jira/browse/SPARK-22372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22372: --- Assignee: Marcelo Vanzin > Make YARN client extend SparkApplication >

[jira] [Commented] (SPARK-24016) Yarn does not update node blacklist in static allocation

[ https://issues.apache.org/jira/browse/SPARK-24016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445252#comment-16445252 ] Saisai Shao commented on SPARK-24016: - I think this can be useful if we enabled

[jira] [Commented] (SPARK-24008) SQL/Hive Context fails with NullPointerException

[ https://issues.apache.org/jira/browse/SPARK-24008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443495#comment-16443495 ] Saisai Shao commented on SPARK-24008: - The case you provided seems not so valid. You're trying to

[jira] [Commented] (SPARK-24008) SQL/Hive Context fails with NullPointerException

[ https://issues.apache.org/jira/browse/SPARK-24008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443475#comment-16443475 ] Saisai Shao commented on SPARK-24008: - Why do you need to broadcast {{SQLContext}} or

[jira] [Resolved] (SPARK-24014) Add onStreamingStarted method to StreamingListener

[ https://issues.apache.org/jira/browse/SPARK-24014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24014. - Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by pull

[jira] [Assigned] (SPARK-24014) Add onStreamingStarted method to StreamingListener

[ https://issues.apache.org/jira/browse/SPARK-24014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24014: --- Assignee: Liang-Chi Hsieh > Add onStreamingStarted method to StreamingListener >

[jira] [Commented] (SPARK-23989) When using `SortShuffleWriter`, the data will be overwritten

[ https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441957#comment-16441957 ] Saisai Shao commented on SPARK-23989: - I've no idea what are you trying to express. What specific

[jira] [Commented] (SPARK-23843) Deploy yarn meets incorrect LOCALIZED_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-23843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441930#comment-16441930 ] Saisai Shao commented on SPARK-23843: - I think this issue is due to your "new Hadoop-compatible

[jira] [Commented] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441919#comment-16441919 ] Saisai Shao commented on SPARK-23830: - What is the reason to use {{class}} instead of {{object}},

[jira] [Commented] (SPARK-24001) Multinode cluster

[ https://issues.apache.org/jira/browse/SPARK-24001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441918#comment-16441918 ] Saisai Shao commented on SPARK-24001: - Question should go to mail list. > Multinode cluster >

[jira] [Resolved] (SPARK-24001) Multinode cluster

[ https://issues.apache.org/jira/browse/SPARK-24001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24001. - Resolution: Invalid > Multinode cluster > -- > > Key:

[jira] [Commented] (SPARK-23989) When using `SortShuffleWriter`, the data will be overwritten

[ https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441906#comment-16441906 ] Saisai Shao commented on SPARK-23989: - Please provide a reproducible case. Did you reuse the object

[jira] [Commented] (SPARK-23982) NoSuchMethodException: There is no startCredentialUpdater method in the object YarnSparkHadoopUtil

[ https://issues.apache.org/jira/browse/SPARK-23982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441881#comment-16441881 ] Saisai Shao commented on SPARK-23982: - This method should be existed. Would you please paste your

[jira] [Resolved] (SPARK-23956) Use effective RPC port in AM registration

[ https://issues.apache.org/jira/browse/SPARK-23956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23956. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21047

[jira] [Assigned] (SPARK-23956) Use effective RPC port in AM registration

[ https://issues.apache.org/jira/browse/SPARK-23956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23956: --- Assignee: Gera Shegalov > Use effective RPC port in AM registration >

<    1   2   3   4   5   6   7   8   9   10   >