[jira] [Updated] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

2018-09-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22357: Labels: behavior-changes (was: ) > SparkContext.binaryFiles ignore minPartitions parameter >

[jira] [Updated] (SPARK-25312) Add description for the conf spark.network.crypto.keyFactoryIterations

2018-09-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25312: Labels: starter (was: ) > Add description for the conf spark.network.crypto.keyFactoryIterations >

[jira] [Created] (SPARK-25312) Add description for the conf spark.network.crypto.keyFactoryIterations

2018-09-02 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25312: --- Summary: Add description for the conf spark.network.crypto.keyFactoryIterations Key: SPARK-25312 URL: https://issues.apache.org/jira/browse/SPARK-25312 Project: Spark

[jira] [Commented] (SPARK-25150) Joining DataFrames derived from the same source yields confusing/incorrect results

2018-09-02 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601786#comment-16601786 ] Peter Toth commented on SPARK-25150: [~EeveeB], sorry, I have just noticed that you might have

[jira] [Commented] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

2018-09-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601777#comment-16601777 ] Apache Spark commented on SPARK-25044: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Updated] (SPARK-25301) When a view uses an UDF from a non default database, Spark analyser throws AnalysisException

2018-09-02 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-25301: - Description: When a hive view uses an UDF from a non default database, Spark analyser throws

[jira] [Commented] (SPARK-25311) `SPARK_LOCAL_HOSTNAME` unsupport IPV6 when do host checking

2018-09-02 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601751#comment-16601751 ] Xiaochen Ouyang commented on SPARK-25311: - IPV6, IPV4 regular expression can be used to solve

[jira] [Created] (SPARK-25311) `SPARK_LOCAL_HOSTNAME` unsupport IPV6 when do host checking

2018-09-02 Thread Xiaochen Ouyang (JIRA)
Xiaochen Ouyang created SPARK-25311: --- Summary: `SPARK_LOCAL_HOSTNAME` unsupport IPV6 when do host checking Key: SPARK-25311 URL: https://issues.apache.org/jira/browse/SPARK-25311 Project: Spark

[jira] [Assigned] (SPARK-25304) enable HiveSparkSubmitSuite SPARK-8489 test for Scala 2.12

2018-09-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25304: - Assignee: Darcy Shen > enable HiveSparkSubmitSuite SPARK-8489 test for Scala 2.12 >

[jira] [Resolved] (SPARK-25304) enable HiveSparkSubmitSuite SPARK-8489 test for Scala 2.12

2018-09-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25304. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22308

[jira] [Updated] (SPARK-25304) enable HiveSparkSubmitSuite SPARK-8489 test for Scala 2.12

2018-09-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25304: -- Priority: Minor (was: Major) > enable HiveSparkSubmitSuite SPARK-8489 test for Scala 2.12 >

[jira] [Commented] (SPARK-23253) Only write shuffle temporary index file when there is not an existing one

2018-09-02 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601747#comment-16601747 ] Wenchen Fan commented on SPARK-23253: - Hi [~irashid] thanks for providing these references and sorry

[jira] [Assigned] (SPARK-25150) Joining DataFrames derived from the same source yields confusing/incorrect results

2018-09-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25150: Assignee: (was: Apache Spark) > Joining DataFrames derived from the same source

[jira] [Commented] (SPARK-25150) Joining DataFrames derived from the same source yields confusing/incorrect results

2018-09-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601734#comment-16601734 ] Apache Spark commented on SPARK-25150: -- User 'peter-toth' has created a pull request for this

[jira] [Assigned] (SPARK-25150) Joining DataFrames derived from the same source yields confusing/incorrect results

2018-09-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25150: Assignee: Apache Spark > Joining DataFrames derived from the same source yields

[jira] [Commented] (SPARK-25135) insert datasource table may all null when select from view on parquet

2018-09-02 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601728#comment-16601728 ] Yuming Wang commented on SPARK-25135: - [https://github.com/apache/spark/pull/22311]

[jira] [Updated] (SPARK-25265) Fix memory leak in Barrier Execution Mode

2018-09-02 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-25265: --- Summary: Fix memory leak in Barrier Execution Mode (was: Fix memory leak vulnerability in

[jira] [Resolved] (SPARK-25265) Fix memory leak vulnerability in Barrier Execution Mode

2018-09-02 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-25265. Resolution: Duplicate Thanks for the notification. It may be accidentally duplicated.

[jira] [Commented] (SPARK-25176) Kryo fails to serialize a parametrised type hierarchy

2018-09-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601643#comment-16601643 ] Dongjoon Hyun commented on SPARK-25176: --- Sorry, [~m.pryahin]. I overlooked the example in [4]. I

[jira] [Issue Comment Deleted] (SPARK-25176) Kryo fails to serialize a parametrised type hierarchy

2018-09-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25176: -- Comment: was deleted (was: [~m.pryahin]. There is not much information for this. Since this

[jira] [Commented] (SPARK-25176) Kryo fails to serialize a parametrised type hierarchy

2018-09-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601642#comment-16601642 ] Dongjoon Hyun commented on SPARK-25176: --- [~m.pryahin]. There is not much information for this.

[jira] [Commented] (SPARK-20389) Upgrade kryo to fix NegativeArraySizeException

2018-09-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601640#comment-16601640 ] Dongjoon Hyun commented on SPARK-20389: --- Hi, [~georg.kf.hei...@gmail.com] and [~tashoyan].

[jira] [Commented] (SPARK-25310) ArraysOverlap may throw a CompileException

2018-09-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601260#comment-16601260 ] Apache Spark commented on SPARK-25310: -- User 'kiszk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25310) ArraysOverlap may throw a CompileException

2018-09-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25310: Assignee: Apache Spark > ArraysOverlap may throw a CompileException >

[jira] [Assigned] (SPARK-25310) ArraysOverlap may throw a CompileException

2018-09-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25310: Assignee: (was: Apache Spark) > ArraysOverlap may throw a CompileException >

[jira] [Updated] (SPARK-25310) ArraysOverlap may throw a CompileException

2018-09-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25310: - Description: Invoking {{ArraysOverlap}} function with non-nullable array type throws

[jira] [Updated] (SPARK-25310) ArraysOverlap may throw a CompileException

2018-09-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25310: - Summary: ArraysOverlap may throw a CompileException (was: ArraysOverlap throws an

[jira] [Created] (SPARK-25310) ArraysOverlap throws an Exception

2018-09-02 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-25310: Summary: ArraysOverlap throws an Exception Key: SPARK-25310 URL: https://issues.apache.org/jira/browse/SPARK-25310 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-25309) Sci-kit Learn like Auto Pipeline Parallelization in Spark

2018-09-02 Thread Ravi (JIRA)
Ravi created SPARK-25309: Summary: Sci-kit Learn like Auto Pipeline Parallelization in Spark Key: SPARK-25309 URL: https://issues.apache.org/jira/browse/SPARK-25309 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25048) Pivoting by multiple columns in Scala/Java

2018-09-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601210#comment-16601210 ] Apache Spark commented on SPARK-25048: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Resolved] (SPARK-25007) Add array_intersect / array_except /array_union / array_shuffle to SparkR

2018-09-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-25007. -- Resolution: Fixed Fix Version/s: 2.4.0 > Add array_intersect / array_except

[jira] [Assigned] (SPARK-25007) Add array_intersect / array_except /array_union / array_shuffle to SparkR

2018-09-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-25007: Assignee: Huaxin Gao > Add array_intersect / array_except /array_union / array_shuffle