[jira] [Commented] (SPARK-24815) Structured Streaming should support dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-24815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853745#comment-16853745 ] Karthik Palaniappan commented on SPARK-24815: - Just to clarify: does Spark allow having

[jira] [Comment Edited] (SPARK-24815) Structured Streaming should support dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-24815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852541#comment-16852541 ] Karthik Palaniappan edited comment on SPARK-24815 at 5/31/19 2:26 AM:

[jira] [Commented] (SPARK-24815) Structured Streaming should support dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-24815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852541#comment-16852541 ] Karthik Palaniappan commented on SPARK-24815: - I was starting to update the JIRA description

[jira] [Updated] (SPARK-24815) Structured Streaming should support dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-24815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Palaniappan updated SPARK-24815: Description: For batch jobs, dynamic allocation is very useful for adding and

[jira] [Commented] (SPARK-24815) Structured Streaming should support dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-24815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849968#comment-16849968 ] Karthik Palaniappan commented on SPARK-24815: - Since there's discussion about this FR on the

[jira] [Commented] (SPARK-24668) PySpark crashes when getting the webui url if the webui is disabled

[ https://issues.apache.org/jira/browse/SPARK-24668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796263#comment-16796263 ] Karthik Palaniappan commented on SPARK-24668: - Please take it :) I didn't have time to work

[jira] [Created] (SPARK-24815) Structured Streaming should support dynamic allocation

Karthik Palaniappan created SPARK-24815: --- Summary: Structured Streaming should support dynamic allocation Key: SPARK-24815 URL: https://issues.apache.org/jira/browse/SPARK-24815 Project: Spark

[jira] [Commented] (SPARK-24668) PySpark crashes when getting the webui url if the webui is disabled

[ https://issues.apache.org/jira/browse/SPARK-24668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530344#comment-16530344 ] Karthik Palaniappan commented on SPARK-24668: - Sounds good to me – I'll work on implementing

[jira] [Updated] (SPARK-24668) PySpark crashes when getting the webui url if the webui is disabled

[ https://issues.apache.org/jira/browse/SPARK-24668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Palaniappan updated SPARK-24668: Environment: * Spark 2.3.0 * Spark-on-YARN * Java 8 * Python 3.6.5 * Jupyter

[jira] [Created] (SPARK-24668) PySpark crashes when getting the webui url if the webui is disabled

Karthik Palaniappan created SPARK-24668: --- Summary: PySpark crashes when getting the webui url if the webui is disabled Key: SPARK-24668 URL: https://issues.apache.org/jira/browse/SPARK-24668

[jira] [Reopened] (SPARK-19941) Spark should not schedule tasks on executors on decommissioning YARN nodes

[ https://issues.apache.org/jira/browse/SPARK-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Palaniappan reopened SPARK-19941: - This should be marked as a duplicate of 

[jira] [Created] (SPARK-21960) Spark Streaming Dynamic Allocation should respect spark.executor.instances

Karthik Palaniappan created SPARK-21960: --- Summary: Spark Streaming Dynamic Allocation should respect spark.executor.instances Key: SPARK-21960 URL: https://issues.apache.org/jira/browse/SPARK-21960

[jira] [Commented] (SPARK-21792) Document Spark Streaming Dynamic Allocation Configuration

[ https://issues.apache.org/jira/browse/SPARK-21792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134329#comment-16134329 ] Karthik Palaniappan commented on SPARK-21792: - The author of the file is [~tdas]. >

[jira] [Created] (SPARK-21792) Document Spark Streaming Dynamic Allocation Configuration

Karthik Palaniappan created SPARK-21792: --- Summary: Document Spark Streaming Dynamic Allocation Configuration Key: SPARK-21792 URL: https://issues.apache.org/jira/browse/SPARK-21792 Project:

[jira] [Commented] (SPARK-19941) Spark should not schedule tasks on executors on decommissioning YARN nodes

[ https://issues.apache.org/jira/browse/SPARK-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930433#comment-15930433 ] Karthik Palaniappan commented on SPARK-19941: - Yeah, I could have been more clear. The

[jira] [Comment Edited] (SPARK-19941) Spark should not schedule tasks on executors on decommissioning YARN nodes

[ https://issues.apache.org/jira/browse/SPARK-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930433#comment-15930433 ] Karthik Palaniappan edited comment on SPARK-19941 at 3/17/17 6:28 PM:

[jira] [Commented] (SPARK-19941) Spark should not schedule tasks on executors on decommissioning YARN nodes

[ https://issues.apache.org/jira/browse/SPARK-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923363#comment-15923363 ] Karthik Palaniappan commented on SPARK-19941: - To repro: Set up Spark on YARN (Hadoop 2.8).

[jira] [Created] (SPARK-19941) Spark should not schedule tasks on executors on decommissioning YARN nodes

Karthik Palaniappan created SPARK-19941: --- Summary: Spark should not schedule tasks on executors on decommissioning YARN nodes Key: SPARK-19941 URL: https://issues.apache.org/jira/browse/SPARK-19941