[jira] [Commented] (SPARK-23904) Big execution plan cause OOM

2018-04-14 Thread Izek Greenfield (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438591#comment-16438591 ] Izek Greenfield commented on SPARK-23904: - [~viirya] this does not help because s

[jira] [Updated] (SPARK-23984) PySpark Bindings for K8S

2018-04-14 Thread Ilan Filonenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilan Filonenko updated SPARK-23984: --- Summary: PySpark Bindings for K8S (was: PySpark Bindings) > PySpark Bindings for K8S > -

[jira] [Updated] (SPARK-23984) PySpark Bindings

2018-04-14 Thread Ilan Filonenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilan Filonenko updated SPARK-23984: --- Shepherd: (was: Holden Karau) > PySpark Bindings > > > Key

[jira] [Created] (SPARK-23984) PySpark Bindings

2018-04-14 Thread Ilan Filonenko (JIRA)
Ilan Filonenko created SPARK-23984: -- Summary: PySpark Bindings Key: SPARK-23984 URL: https://issues.apache.org/jira/browse/SPARK-23984 Project: Spark Issue Type: New Feature Compon

[jira] [Assigned] (SPARK-23936) High-order function: map_concat(map1, map2, ..., mapN) → map

2018-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23936: Assignee: (was: Apache Spark) > High-order function: map_concat(map1, map2, ..., mapN)

[jira] [Commented] (SPARK-23936) High-order function: map_concat(map1, map2, ..., mapN) → map

2018-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438576#comment-16438576 ] Apache Spark commented on SPARK-23936: -- User 'bersprockets' has created a pull reque

[jira] [Assigned] (SPARK-23936) High-order function: map_concat(map1, map2, ..., mapN) → map

2018-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23936: Assignee: Apache Spark > High-order function: map_concat(map1, map2, ..., mapN) → > map >

[jira] [Commented] (SPARK-23982) NoSuchMethodException: There is no startCredentialUpdater method in the object YarnSparkHadoopUtil

2018-04-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438573#comment-16438573 ] Marcelo Vanzin commented on SPARK-23982: More details? https://github.com/apache

[jira] [Commented] (SPARK-23983) Disable X-Frame-Options from Spark UI response headers if explicitly configured

2018-04-14 Thread Joe Pallas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438554#comment-16438554 ] Joe Pallas commented on SPARK-23983: It seems clear that the CSP frame-ancestors appr

[jira] [Commented] (SPARK-21163) DataFrame.toPandas should respect the data type

2018-04-14 Thread Ed Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438535#comment-16438535 ] Ed Lee commented on SPARK-21163: Had a question: in Spark 2.2.1, if I do a .toPandas on a

[jira] [Updated] (SPARK-23983) Disable X-Frame-Options from Spark UI response headers if explicitly configured

2018-04-14 Thread Taylor Cressy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taylor Cressy updated SPARK-23983: -- Description: We should introduce a configuration for the spark UI to omit X-Frame-Options from

[jira] [Updated] (SPARK-23983) Disable X-Frame-Options from Spark UI response headers if explicitly configured

2018-04-14 Thread Taylor Cressy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taylor Cressy updated SPARK-23983: -- Description: We should introduce a configuration for the spark UI to omit X-Frame-Options from

[jira] [Created] (SPARK-23983) Disable X-Frame-Options from Spark UI response headers if explicitly configured

2018-04-14 Thread Taylor Cressy (JIRA)
Taylor Cressy created SPARK-23983: - Summary: Disable X-Frame-Options from Spark UI response headers if explicitly configured Key: SPARK-23983 URL: https://issues.apache.org/jira/browse/SPARK-23983 Pro

[jira] [Commented] (SPARK-23891) Debian based Dockerfile

2018-04-14 Thread Sercan Karaoglu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438509#comment-16438509 ] Sercan Karaoglu commented on SPARK-23891: - So to summarize, as a user I want to h

[jira] [Commented] (SPARK-23891) Debian based Dockerfile

2018-04-14 Thread Sercan Karaoglu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438500#comment-16438500 ] Sercan Karaoglu commented on SPARK-23891: - I don't know if you guys want to do th

[jira] [Commented] (SPARK-23891) Debian based Dockerfile

2018-04-14 Thread Sercan Karaoglu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438499#comment-16438499 ] Sercan Karaoglu commented on SPARK-23891: - Sure! I've just attached it, and as a

[jira] [Updated] (SPARK-23891) Debian based Dockerfile

2018-04-14 Thread Sercan Karaoglu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sercan Karaoglu updated SPARK-23891: Attachment: (was: Dockerfile) > Debian based Dockerfile > --- > >

[jira] [Updated] (SPARK-23891) Debian based Dockerfile

2018-04-14 Thread Sercan Karaoglu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sercan Karaoglu updated SPARK-23891: Attachment: Dockerfile > Debian based Dockerfile > --- > >

[jira] [Updated] (SPARK-23891) Debian based Dockerfile

2018-04-14 Thread Sercan Karaoglu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sercan Karaoglu updated SPARK-23891: Attachment: Dockerfile > Debian based Dockerfile > --- > >

[jira] [Commented] (SPARK-23891) Debian based Dockerfile

2018-04-14 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438410#comment-16438410 ] Erik Erlandson commented on SPARK-23891: [~SercanKaraoglu] thanks for the informa

[jira] [Resolved] (SPARK-23970) pyspark - simple filter/select doesn't use all tasks when coalesce is set

2018-04-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23970. -- Resolution: Not A Problem > pyspark - simple filter/select doesn't use all tasks when coalesce

[jira] [Commented] (SPARK-23970) pyspark - simple filter/select doesn't use all tasks when coalesce is set

2018-04-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438376#comment-16438376 ] Hyukjin Kwon commented on SPARK-23970: -- It's clearly documented. Let's leave this re

[jira] [Commented] (SPARK-23982) NoSuchMethodException: There is no startCredentialUpdater method in the object YarnSparkHadoopUtil

2018-04-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438350#comment-16438350 ] Hyukjin Kwon commented on SPARK-23982: -- Don't set a blocker which is usually reserve

[jira] [Updated] (SPARK-23982) NoSuchMethodException: There is no startCredentialUpdater method in the object YarnSparkHadoopUtil

2018-04-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23982: - Priority: Major (was: Blocker) > NoSuchMethodException: There is no startCredentialUpdater metho

[jira] [Commented] (SPARK-23896) Improve PartitioningAwareFileIndex

2018-04-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438346#comment-16438346 ] Hyukjin Kwon commented on SPARK-23896: -- (let's avoid to describe the JIRA title just

[jira] [Updated] (SPARK-23942) PySpark's collect doesn't trigger QueryExecutionListener

2018-04-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23942: - Fix Version/s: 2.3.1 > PySpark's collect doesn't trigger QueryExecutionListener > ---

[jira] [Commented] (SPARK-23973) Remove consecutive sorts

2018-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438314#comment-16438314 ] Apache Spark commented on SPARK-23973: -- User 'mgaido91' has created a pull request f

[jira] [Assigned] (SPARK-23973) Remove consecutive sorts

2018-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23973: Assignee: Apache Spark > Remove consecutive sorts > > >

[jira] [Assigned] (SPARK-23973) Remove consecutive sorts

2018-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23973: Assignee: (was: Apache Spark) > Remove consecutive sorts > >

[jira] [Created] (SPARK-23982) NoSuchMethodException: There is no startCredentialUpdater method in the object YarnSparkHadoopUtil

2018-04-14 Thread John (JIRA)
John created SPARK-23982: Summary: NoSuchMethodException: There is no startCredentialUpdater method in the object YarnSparkHadoopUtil Key: SPARK-23982 URL: https://issues.apache.org/jira/browse/SPARK-23982 Pr