[jira] [Commented] (SPARK-22623) Getting TimeoutException In My Spark Application

2017-11-27 Thread Sanchay Dewan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268345#comment-16268345 ] Sanchay Dewan commented on SPARK-22623: --- The issue was I still had .setMaster("local") in my code,

[jira] [Updated] (SPARK-22622) OutOfMemory thrown by Closure Serializer

2017-11-27 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghavendra updated SPARK-22622: Priority: Critical (was: Blocker) > OutOfMemory thrown by Closure Serializer >

[jira] [Created] (SPARK-22624) Expose range partitioning shuffle introduced by SPARK-22614

2017-11-27 Thread Adrian Ionescu (JIRA)
Adrian Ionescu created SPARK-22624: -- Summary: Expose range partitioning shuffle introduced by SPARK-22614 Key: SPARK-22624 URL: https://issues.apache.org/jira/browse/SPARK-22624 Project: Spark

[jira] [Updated] (SPARK-22622) OutOfMemory thrown by Closure Serializer

2017-11-27 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghavendra updated SPARK-22622: Description: While moving from a Stage to another, the Closure serializer is trying to Serialize

[jira] [Updated] (SPARK-22622) OutOfMemory thrown by Closure Serializer

2017-11-27 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghavendra updated SPARK-22622: Description: While moving from a Stage to another, the Closure serializer is trying to Serialize

[jira] [Updated] (SPARK-22622) OutOfMemory thrown by Closure Serializer

2017-11-27 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghavendra updated SPARK-22622: Description: While moving from a Stage to another, the Closure serializer is trying to Serialize

[jira] [Created] (SPARK-22623) Getting TimeoutException In My Spark Application

2017-11-27 Thread Sanchay Dewan (JIRA)
Sanchay Dewan created SPARK-22623: - Summary: Getting TimeoutException In My Spark Application Key: SPARK-22623 URL: https://issues.apache.org/jira/browse/SPARK-22623 Project: Spark Issue

[jira] [Created] (SPARK-22622) OutOfMemory thrown by Closure Serializer

2017-11-27 Thread Raghavendra (JIRA)
Raghavendra created SPARK-22622: --- Summary: OutOfMemory thrown by Closure Serializer Key: SPARK-22622 URL: https://issues.apache.org/jira/browse/SPARK-22622 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-22621) Upgrade maven enforcer plugin to 3.0.0-M1 to compile with JDK9

2017-11-27 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang resolved SPARK-22621. Resolution: Duplicate > Upgrade maven enforcer plugin to 3.0.0-M1 to compile with JDK9 >

[jira] [Created] (SPARK-22621) Upgrade maven enforcer plugin to 3.0.0-M1 to compile with JDK9

2017-11-27 Thread liyunzhang (JIRA)
liyunzhang created SPARK-22621: -- Summary: Upgrade maven enforcer plugin to 3.0.0-M1 to compile with JDK9 Key: SPARK-22621 URL: https://issues.apache.org/jira/browse/SPARK-22621 Project: Spark

[jira] [Commented] (SPARK-21497) Pull non-deterministic joining keys from Join operator

2017-11-27 Thread a283791411 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268033#comment-16268033 ] a283791411 commented on SPARK-21497: This feature is very useful. Please review. > Pull

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-27 Thread Min Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267948#comment-16267948 ] Min Shen commented on SPARK-22373: -- The code that would throw this NPE looks like the following:

[jira] [Assigned] (SPARK-22484) PySpark DataFrame.write.csv(quote="") uses nullchar as quote

2017-11-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22484: Assignee: John Bollenbacher > PySpark DataFrame.write.csv(quote="") uses nullchar as

[jira] [Resolved] (SPARK-22484) PySpark DataFrame.write.csv(quote="") uses nullchar as quote

2017-11-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22484. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19814

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267901#comment-16267901 ] Kazuaki Ishizaki commented on SPARK-22373: -- [~mshen] Thank you for reporting this issue. We

[jira] [Assigned] (SPARK-22520) Support code generation also for complex CASE WHEN

2017-11-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22520: --- Assignee: Marco Gaido > Support code generation also for complex CASE WHEN >

[jira] [Resolved] (SPARK-22520) Support code generation also for complex CASE WHEN

2017-11-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22520. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19752

[jira] [Commented] (SPARK-20050) Kafka 0.10 DirectStream doesn't commit last processed batch's offset when graceful shutdown

2017-11-27 Thread Rohit Thadani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267802#comment-16267802 ] Rohit Thadani commented on SPARK-20050: --- I see the bug has been reported against kafka 2.2.0 will

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-27 Thread Min Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267723#comment-16267723 ] Min Shen commented on SPARK-22373: -- Also facing this issue. >From our experience, it is more likely to

[jira] [Created] (SPARK-22620) Deadlock in blacklisting when executor is removed due to a heartbeat timeout

2017-11-27 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-22620: - Summary: Deadlock in blacklisting when executor is removed due to a heartbeat timeout Key: SPARK-22620 URL: https://issues.apache.org/jira/browse/SPARK-22620

[jira] [Commented] (SPARK-22541) Dataframes: applying multiple filters one after another using udfs and accumulators results in faulty accumulators

2017-11-27 Thread Andrew Snare (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267567#comment-16267567 ] Andrew Snare commented on SPARK-22541: -- Although this ship has sailed, a search brought me here

[jira] [Commented] (SPARK-22619) Implement the CG method for ALS

2017-11-27 Thread Nathaniel Wendt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267270#comment-16267270 ] Nathaniel Wendt commented on SPARK-22619: - I've implemented the algorithm and tested it's

[jira] [Created] (SPARK-22619) Implement the CG method for ALS

2017-11-27 Thread Nathaniel Wendt (JIRA)
Nathaniel Wendt created SPARK-22619: --- Summary: Implement the CG method for ALS Key: SPARK-22619 URL: https://issues.apache.org/jira/browse/SPARK-22619 Project: Spark Issue Type:

[jira] [Commented] (SPARK-22618) RDD.unpersist can cause fatal exception when used with dynamic allocation

2017-11-27 Thread Brad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267223#comment-16267223 ] Brad commented on SPARK-22618: -- I have a PR for this forthcoming. > RDD.unpersist can cause fatal exception

[jira] [Created] (SPARK-22618) RDD.unpersist can cause fatal exception when used with dynamic allocation

2017-11-27 Thread Brad (JIRA)
Brad created SPARK-22618: Summary: RDD.unpersist can cause fatal exception when used with dynamic allocation Key: SPARK-22618 URL: https://issues.apache.org/jira/browse/SPARK-22618 Project: Spark

[jira] [Commented] (SPARK-22051) Explicit control of number of partitions after dataframe operations (join, order...)

2017-11-27 Thread Fernando Pereira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267169#comment-16267169 ] Fernando Pereira commented on SPARK-22051: -- Ideas anyone? > Explicit control of number of

[jira] [Resolved] (SPARK-22529) Relation stats should be consistent with other plans based on cbo config

2017-11-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22529. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19757

[jira] [Assigned] (SPARK-22529) Relation stats should be consistent with other plans based on cbo config

2017-11-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22529: --- Assignee: Zhenhua Wang > Relation stats should be consistent with other plans based on cbo

[jira] [Commented] (SPARK-22616) df.cache() / df.persist() should have an option blocking like df.unpersist()

2017-11-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267017#comment-16267017 ] Sean Owen commented on SPARK-22616: --- The method signature becomes different in the Java bytecode. Try

[jira] [Commented] (SPARK-22526) Document closing of PortableDataInputStream in binaryFiles

2017-11-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267015#comment-16267015 ] Steve Loughran commented on SPARK-22526: HADOOP-15071 updates the s3a troubleshooting with what

[jira] [Commented] (SPARK-22526) Document closing of PortableDataInputStream in binaryFiles

2017-11-27 Thread mohamed imran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267006#comment-16267006 ] mohamed imran commented on SPARK-22526: --- [~srowen] Sorry didn't get a chance to work! i will

[jira] [Commented] (SPARK-22616) df.cache() / df.persist() should have an option blocking like df.unpersist()

2017-11-27 Thread Andreas Maier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266995#comment-16266995 ] Andreas Maier commented on SPARK-22616: --- I don't see how simply adding an option "blocking" with

[jira] [Updated] (SPARK-22614) Expose range partitioning shuffle

2017-11-27 Thread Adrian Ionescu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Ionescu updated SPARK-22614: --- Description: Right now, the Dataset API only offers two possibilities for explicitly

[jira] [Assigned] (SPARK-22614) Expose range partitioning shuffle

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22614: Assignee: Apache Spark > Expose range partitioning shuffle >

[jira] [Commented] (SPARK-22614) Expose range partitioning shuffle

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266975#comment-16266975 ] Apache Spark commented on SPARK-22614: -- User 'adrian-ionescu' has created a pull request for this

[jira] [Assigned] (SPARK-22614) Expose range partitioning shuffle

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22614: Assignee: (was: Apache Spark) > Expose range partitioning shuffle >

[jira] [Commented] (SPARK-22611) Spark Kinesis ProvisionedThroughputExceededException leads to dropped records

2017-11-27 Thread Richard Moorhead (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266971#comment-16266971 ] Richard Moorhead commented on SPARK-22611: -- Further investigation seems to indicate that setting

[jira] [Assigned] (SPARK-22617) make splitExpressions extract current input of the context

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22617: Assignee: Apache Spark (was: Wenchen Fan) > make splitExpressions extract current input

[jira] [Commented] (SPARK-22617) make splitExpressions extract current input of the context

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266955#comment-16266955 ] Apache Spark commented on SPARK-22617: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22617) make splitExpressions extract current input of the context

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22617: Assignee: Wenchen Fan (was: Apache Spark) > make splitExpressions extract current input

[jira] [Commented] (SPARK-22585) Url encoding of jar path expected?

2017-11-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266941#comment-16266941 ] Sean Owen commented on SPARK-22585: --- URI encoding applies to all of those URI schemes. I don't think

[jira] [Created] (SPARK-22617) make splitExpressions extract current input of the context

2017-11-27 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-22617: --- Summary: make splitExpressions extract current input of the context Key: SPARK-22617 URL: https://issues.apache.org/jira/browse/SPARK-22617 Project: Spark

[jira] [Commented] (SPARK-22616) df.cache() / df.persist() should have an option blocking like df.unpersist()

2017-11-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266920#comment-16266920 ] Sean Owen commented on SPARK-22616: --- I do see this come up so much that I think it's a valid feature

[jira] [Created] (SPARK-22616) df.cache() / df.persist() should have an option blocking like df.unpersist()

2017-11-27 Thread Andreas Maier (JIRA)
Andreas Maier created SPARK-22616: - Summary: df.cache() / df.persist() should have an option blocking like df.unpersist() Key: SPARK-22616 URL: https://issues.apache.org/jira/browse/SPARK-22616

[jira] [Issue Comment Deleted] (SPARK-22585) Url encoding of jar path expected?

2017-11-27 Thread Jakub Dubovsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakub Dubovsky updated SPARK-22585: --- Comment: was deleted (was: Additional question. Do you think it is real to make this into

[jira] [Commented] (SPARK-22526) Document closing of PortableDataInputStream in binaryFiles

2017-11-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266891#comment-16266891 ] Sean Owen commented on SPARK-22526: --- Ping [~imranece59] > Document closing of PortableDataInputStream

[jira] [Commented] (SPARK-22585) Url encoding of jar path expected?

2017-11-27 Thread Jakub Dubovsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266885#comment-16266885 ] Jakub Dubovsky commented on SPARK-22585: Additional question. Do you think it is real to make

[jira] [Commented] (SPARK-22585) Url encoding of jar path expected?

2017-11-27 Thread Jakub Dubovsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266859#comment-16266859 ] Jakub Dubovsky commented on SPARK-22585: [~srowen] I'd like to understand more about solution you

[jira] [Commented] (SPARK-22612) NullPointerException in AppendOnlyMap

2017-11-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266834#comment-16266834 ] Sean Owen commented on SPARK-22612: --- It sounds like one element of a tuple used as a key is null? what

[jira] [Assigned] (SPARK-22428) Document spark properties for configuring the ContextCleaner

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22428: Assignee: (was: Apache Spark) > Document spark properties for configuring the

[jira] [Commented] (SPARK-22428) Document spark properties for configuring the ContextCleaner

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266821#comment-16266821 ] Apache Spark commented on SPARK-22428: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Assigned] (SPARK-22428) Document spark properties for configuring the ContextCleaner

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22428: Assignee: Apache Spark > Document spark properties for configuring the ContextCleaner >

[jira] [Updated] (SPARK-22615) Handle more cases in PropagateEmptyRelation

2017-11-27 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-22615: --- Description: Currently, in the optimize rule `PropagateEmptyRelation`, the following cases

[jira] [Updated] (SPARK-22615) Handle more cases in PropagateEmptyRelation

2017-11-27 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-22615: --- Description: Currently, in the optimize rule `PropagateEmptyRelation`, the following cases

[jira] [Resolved] (SPARK-10969) Spark Streaming Kinesis: Allow specifying separate credentials for Kinesis and DynamoDB

2017-11-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10969. --- Resolution: Duplicate > Spark Streaming Kinesis: Allow specifying separate credentials for Kinesis

[jira] [Commented] (SPARK-22615) Handle more cases in PropagateEmptyRelation

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266745#comment-16266745 ] Apache Spark commented on SPARK-22615: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-22615) Handle more cases in PropagateEmptyRelation

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22615: Assignee: (was: Apache Spark) > Handle more cases in PropagateEmptyRelation >

[jira] [Assigned] (SPARK-22615) Handle more cases in PropagateEmptyRelation

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22615: Assignee: Apache Spark > Handle more cases in PropagateEmptyRelation >

[jira] [Updated] (SPARK-22615) Handle more cases in PropagateEmptyRelation

2017-11-27 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-22615: --- Description: Currently, in the optimize rule `PropagateEmptyRelation`, the following cases

[jira] [Created] (SPARK-22615) Handle more cases in PropagateEmptyRelation

2017-11-27 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-22615: -- Summary: Handle more cases in PropagateEmptyRelation Key: SPARK-22615 URL: https://issues.apache.org/jira/browse/SPARK-22615 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-22603) 64KB JVM bytecode limit problem with FormatString

2017-11-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22603: --- Assignee: Kazuaki Ishizaki > 64KB JVM bytecode limit problem with FormatString >

[jira] [Resolved] (SPARK-22603) 64KB JVM bytecode limit problem with FormatString

2017-11-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22603. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.2 Issue resolved by pull

[jira] [Commented] (SPARK-22587) Spark job fails if fs.defaultFS and application jar are different url

2017-11-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266697#comment-16266697 ] Steve Loughran commented on SPARK-22587: See also

[jira] [Commented] (SPARK-17901) NettyRpcEndpointRef: Error sending message and Caused by: java.util.ConcurrentModificationException

2017-11-27 Thread Tobias Wolfertshofer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266690#comment-16266690 ] Tobias Wolfertshofer commented on SPARK-17901: -- [~srowen] FWIW we can still reproduce this

[jira] [Created] (SPARK-22614) Expose range partitioning shuffle

2017-11-27 Thread Adrian Ionescu (JIRA)
Adrian Ionescu created SPARK-22614: -- Summary: Expose range partitioning shuffle Key: SPARK-22614 URL: https://issues.apache.org/jira/browse/SPARK-22614 Project: Spark Issue Type:

[jira] [Commented] (SPARK-22587) Spark job fails if fs.defaultFS and application jar are different url

2017-11-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266674#comment-16266674 ] Steve Loughran commented on SPARK-22587: Jerry had already pulled me in for this; it's one of

[jira] [Commented] (SPARK-18905) Potential Issue of Semantics of BatchCompleted

2017-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266655#comment-16266655 ] Apache Spark commented on SPARK-18905: -- User 'victor-wong' has created a pull request for this

[jira] [Created] (SPARK-22613) Make UNCACHE TABLE behaviour consistent with CACHE TABLE

2017-11-27 Thread Andreas Maier (JIRA)
Andreas Maier created SPARK-22613: - Summary: Make UNCACHE TABLE behaviour consistent with CACHE TABLE Key: SPARK-22613 URL: https://issues.apache.org/jira/browse/SPARK-22613 Project: Spark

[jira] [Commented] (SPARK-22250) Be less restrictive on type checking

2017-11-27 Thread Fernando Pereira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266642#comment-16266642 ] Fernando Pereira commented on SPARK-22250: -- [~bryanc] It could help, but it doesn't solve the

[jira] [Commented] (SPARK-19268) File does not exist: /tmp/temporary-157b89c1-27bb-49f3-a70c-ca1b75022b4d/state/0/2/1.delta

2017-11-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266544#comment-16266544 ] Marco Gaido commented on SPARK-19268: - [~zsxwing] I am hitting this too and I am running 2.2.0. My

[jira] [Commented] (SPARK-22428) Document spark properties for configuring the ContextCleaner

2017-11-27 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266521#comment-16266521 ] Gabor Somogyi commented on SPARK-22428: --- I would like to solve this ticket. Please notify me if