[jira] [Created] (SPARK-24122) Allow automatic driver restarts on K8s

Oz Ben-Ami created SPARK-24122: -- Summary: Allow automatic driver restarts on K8s Key: SPARK-24122 URL: https://issues.apache.org/jira/browse/SPARK-24122 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348617#comment-16348617 ] Oz Ben-Ami commented on SPARK-22575: That makes sense. Not sure why we only saw this with STS, likely

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348555#comment-16348555 ] Oz Ben-Ami commented on SPARK-22575: [~mgaido] The issue was apparently related to dynamic allocation

[jira] [Updated] (SPARK-23082) Allow separate node selectors for driver and executors in Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-23082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oz Ben-Ami updated SPARK-23082: --- Description: In YARN, we can use {{spark.yarn.am.nodeLabelExpression}} to submit the Spark driver

[jira] [Updated] (SPARK-23082) Allow separate node selectors for driver and executors in Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-23082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oz Ben-Ami updated SPARK-23082: --- Description: In YARN, we can use spark.yarn.am.nodeLabelExpression to submit the Spark driver to a

[jira] [Created] (SPARK-23082) Allow separate node selectors for driver and executors in Kubernetes

Oz Ben-Ami created SPARK-23082: -- Summary: Allow separate node selectors for driver and executors in Kubernetes Key: SPARK-23082 URL: https://issues.apache.org/jira/browse/SPARK-23082 Project: Spark

[jira] [Commented] (SPARK-23078) Allow Submitting Spark Thrift Server in Cluster Mode

[ https://issues.apache.org/jira/browse/SPARK-23078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16326390#comment-16326390 ] Oz Ben-Ami commented on SPARK-23078: [~mgaido] no objections, Kubernetes is the one that most needs

[jira] [Comment Edited] (SPARK-23078) Allow Submitting Spark Thrift Server in Cluster Mode

[ https://issues.apache.org/jira/browse/SPARK-23078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16326366#comment-16326366 ] Oz Ben-Ami edited comment on SPARK-23078 at 1/15/18 4:02 PM: - [~mgaido] In

[jira] [Commented] (SPARK-23078) Allow Submitting Spark Thrift Server in Cluster Mode

[ https://issues.apache.org/jira/browse/SPARK-23078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16326366#comment-16326366 ] Oz Ben-Ami commented on SPARK-23078: [~mgaido] In Kubernetes you can just create a Service which

[jira] [Created] (SPARK-23078) Allow Submitting Spark Thrift Server in Cluster Mode

Oz Ben-Ami created SPARK-23078: -- Summary: Allow Submitting Spark Thrift Server in Cluster Mode Key: SPARK-23078 URL: https://issues.apache.org/jira/browse/SPARK-23078 Project: Spark Issue Type:

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16262724#comment-16262724 ] Oz Ben-Ami commented on SPARK-22575: Thanks [~mgaido], we are not caching anything with {{CACHE

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261808#comment-16261808 ] Oz Ben-Ami commented on SPARK-22575: We are not explicitly caching any tables, only running SQL

[jira] [Created] (SPARK-22575) Making Spark Thrift Server clean up its cache

Oz Ben-Ami created SPARK-22575: -- Summary: Making Spark Thrift Server clean up its cache Key: SPARK-22575 URL: https://issues.apache.org/jira/browse/SPARK-22575 Project: Spark Issue Type:

[jira] [Commented] (SPARK-18580) Use spark.streaming.backpressure.initialRate in DirectKafkaInputDStream

[ https://issues.apache.org/jira/browse/SPARK-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109265#comment-16109265 ] Oz Ben-Ami commented on SPARK-18580: +1 We're using our own dynamic allocation to scale with incoming

[jira] [Created] (SPARK-17038) StreamingSource reports metrics for lastCompletedBatch instead of lastReceivedBatch

Oz Ben-Ami created SPARK-17038: -- Summary: StreamingSource reports metrics for lastCompletedBatch instead of lastReceivedBatch Key: SPARK-17038 URL: https://issues.apache.org/jira/browse/SPARK-17038