[jira] [Commented] (SPARK-18696) Upgrade sbt plugins

2016-12-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717449#comment-15717449 ] Dongjoon Hyun commented on SPARK-18696: --- Hi, [~WeiqingYang] This seems to be duplic

[jira] [Commented] (SPARK-18593) JDBCRDD returns incorrect results for filters on CHAR of PostgreSQL

2016-12-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717439#comment-15717439 ] Dongjoon Hyun commented on SPARK-18593: --- This will not be backported. For the discu

[jira] [Resolved] (SPARK-18593) JDBCRDD returns incorrect results for filters on CHAR of PostgreSQL

2016-12-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-18593. --- Resolution: Fixed Fix Version/s: 2.0.0 This was fixed since 2.0.0 by several issue (SP

[jira] [Resolved] (SPARK-18362) Use TextFileFormat in implementation of CSVFileFormat

2016-12-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18362. - Resolution: Fixed Fix Version/s: 2.2.0 > Use TextFileFormat in implementation of CSVFileFo

[jira] [Resolved] (SPARK-18695) Bump master branch version to 2.2.0-SNAPSHOT

2016-12-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18695. - Resolution: Fixed Fix Version/s: 2.2.0 > Bump master branch version to 2.2.0-SNAPSHOT > --

[jira] [Commented] (SPARK-15798) Secondary sort in Dataset/DataFrame

2016-12-02 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717366#comment-15717366 ] koert kuipers commented on SPARK-15798: --- using the implementation in spark-sorted

[jira] [Assigned] (SPARK-18671) Add tests to ensure stability of that all Structured Streaming log formats

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18671: Assignee: (was: Apache Spark) > Add tests to ensure stability of that all Structured S

[jira] [Commented] (SPARK-18671) Add tests to ensure stability of that all Structured Streaming log formats

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717234#comment-15717234 ] Apache Spark commented on SPARK-18671: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-18671) Add tests to ensure stability of that all Structured Streaming log formats

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18671: Assignee: Apache Spark > Add tests to ensure stability of that all Structured Streaming lo

[jira] [Commented] (SPARK-12409) JDBC AND operator push down

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717219#comment-15717219 ] Apache Spark commented on SPARK-12409: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-12387) JDBC IN operator push down

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717220#comment-15717220 ] Apache Spark commented on SPARK-12387: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-12391) JDBC OR operator push down

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717221#comment-15717221 ] Apache Spark commented on SPARK-12391: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-18697) Upgrade sbt plugins

2016-12-02 Thread Weiqing Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717193#comment-15717193 ] Weiqing Yang commented on SPARK-18697: -- Will submit a PR after SPARK-18638 ([PR #16

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

2016-12-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717188#comment-15717188 ] Xiao Li commented on SPARK-18209: - Yeah, the view definition will be much shorter in this

[jira] [Created] (SPARK-18697) Upgrade sbt plugins

2016-12-02 Thread Weiqing Yang (JIRA)
Weiqing Yang created SPARK-18697: Summary: Upgrade sbt plugins Key: SPARK-18697 URL: https://issues.apache.org/jira/browse/SPARK-18697 Project: Spark Issue Type: Improvement Compone

[jira] [Created] (SPARK-18696) Upgrade sbt plugins

2016-12-02 Thread Weiqing Yang (JIRA)
Weiqing Yang created SPARK-18696: Summary: Upgrade sbt plugins Key: SPARK-18696 URL: https://issues.apache.org/jira/browse/SPARK-18696 Project: Spark Issue Type: Improvement Compone

[jira] [Updated] (SPARK-12446) Add unit tests for JDBCRDD internal functions

2016-12-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-12446: Fix Version/s: 1.6.4 > Add unit tests for JDBCRDD internal functions >

[jira] [Comment Edited] (SPARK-18209) More robust view canonicalization without full SQL expansion

2016-12-02 Thread Vish Persaud (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717160#comment-15717160 ] Vish Persaud edited comment on SPARK-18209 at 12/3/16 2:08 AM:

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

2016-12-02 Thread Vish Persaud (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717160#comment-15717160 ] Vish Persaud commented on SPARK-18209: -- Might this change also resolve the following

[jira] [Commented] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

2016-12-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717149#comment-15717149 ] Reynold Xin commented on SPARK-18278: - Is there a way to get this working without the

[jira] [Assigned] (SPARK-18695) Bump master branch version to 2.2.0-SNAPSHOT

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18695: Assignee: Apache Spark (was: Reynold Xin) > Bump master branch version to 2.2.0-SNAPSHOT

[jira] [Commented] (SPARK-18695) Bump master branch version to 2.2.0-SNAPSHOT

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717135#comment-15717135 ] Apache Spark commented on SPARK-18695: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-18695) Bump master branch version to 2.2.0-SNAPSHOT

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18695: Assignee: Reynold Xin (was: Apache Spark) > Bump master branch version to 2.2.0-SNAPSHOT

[jira] [Updated] (SPARK-18695) Bump master branch version to 2.2.0-SNAPSHOT

2016-12-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18695: Summary: Bump master branch version to 2.2.0-SNAPSHOT (was: Bump master branch version to 2.2.0)

[jira] [Created] (SPARK-18695) Bump master branch version to 2.2.0

2016-12-02 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-18695: --- Summary: Bump master branch version to 2.2.0 Key: SPARK-18695 URL: https://issues.apache.org/jira/browse/SPARK-18695 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-18690) Backward compatibility of unbounded frames

2016-12-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18690. - Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.1.0 > Backward co

[jira] [Commented] (SPARK-12446) Add unit tests for JDBCRDD internal functions

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717061#comment-15717061 ] Apache Spark commented on SPARK-12446: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Updated] (SPARK-18694) Add StreamingQuery.explain and exception to Python and fix StreamingQueryException

2016-12-02 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18694: - Priority: Blocker (was: Major) > Add StreamingQuery.explain and exception to Python and fix > S

[jira] [Resolved] (SPARK-18324) ML, Graph 2.1 QA: Programming guide update and migration guide

2016-12-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-18324. --- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 Issue resolved

[jira] [Assigned] (SPARK-18694) Add StreamingQuery.explain and exception to Python and fix StreamingQueryException

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18694: Assignee: Shixiong Zhu (was: Apache Spark) > Add StreamingQuery.explain and exception to

[jira] [Assigned] (SPARK-18694) Add StreamingQuery.explain and exception to Python and fix StreamingQueryException

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18694: Assignee: Apache Spark (was: Shixiong Zhu) > Add StreamingQuery.explain and exception to

[jira] [Commented] (SPARK-18694) Add StreamingQuery.explain and exception to Python and fix StreamingQueryException

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716959#comment-15716959 ] Apache Spark commented on SPARK-18694: -- User 'zsxwing' has created a pull request fo

[jira] [Created] (SPARK-18694) Add StreamingQuery.explain and exception to Python and fix StreamingQueryException

2016-12-02 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-18694: Summary: Add StreamingQuery.explain and exception to Python and fix StreamingQueryException Key: SPARK-18694 URL: https://issues.apache.org/jira/browse/SPARK-18694 Pr

[jira] [Created] (SPARK-18693) BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

2016-12-02 Thread Devesh Parekh (JIRA)
Devesh Parekh created SPARK-18693: - Summary: BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data Key: SPARK-18693 URL: https://issues.apache.org/

[jira] [Updated] (SPARK-16920) Investigate and fix issues introduced in SPARK-15858

2016-12-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-16920: -- Shepherd: Joseph K. Bradley Target Version/s: 2.2.0 > Investigate and fix i

[jira] [Commented] (SPARK-16920) Investigate and fix issues introduced in SPARK-15858

2016-12-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716822#comment-15716822 ] Joseph K. Bradley commented on SPARK-16920: --- Targeting for 2.2. I'm Ok with no

[jira] [Commented] (SPARK-18692) Test Java 8 unidoc build on Jenkins master builder

2016-12-02 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716758#comment-15716758 ] Josh Rosen commented on SPARK-18692: +1; this would be great to have. I believe that

[jira] [Created] (SPARK-18692) Test Java 8 unidoc build on Jenkins master builder

2016-12-02 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18692: - Summary: Test Java 8 unidoc build on Jenkins master builder Key: SPARK-18692 URL: https://issues.apache.org/jira/browse/SPARK-18692 Project: Spark

[jira] [Updated] (SPARK-18691) Spark can hang if a node goes down during a shuffle

2016-12-02 Thread Nicholas Brown (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Brown updated SPARK-18691: --- Attachment: retries.txt > Spark can hang if a node goes down during a shuffle > -

[jira] [Created] (SPARK-18691) Spark can hang if a node goes down during a shuffle

2016-12-02 Thread Nicholas Brown (JIRA)
Nicholas Brown created SPARK-18691: -- Summary: Spark can hang if a node goes down during a shuffle Key: SPARK-18691 URL: https://issues.apache.org/jira/browse/SPARK-18691 Project: Spark Issue

[jira] [Commented] (SPARK-18203) Spark Streaming Kinesis Extension v2.0.1 calls non-existent method

2016-12-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716706#comment-15716706 ] Sean Owen commented on SPARK-18203: --- OK, but that's some problem with the versions you'

[jira] [Commented] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

2016-12-02 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716701#comment-15716701 ] Matt Cheah commented on SPARK-18278: There has also been some discussion about this o

[jira] [Commented] (SPARK-18349) Update R API documentation on ml model summary

2016-12-02 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716699#comment-15716699 ] Miao Wang commented on SPARK-18349: --- [~felixcheung]I can help on this task. I am helpin

[jira] [Assigned] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

2016-12-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-17822: - Assignee: Xiangrui Meng > JVMObjectTracker.objMap may leak JVM objects > ---

[jira] [Commented] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

2016-12-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716690#comment-15716690 ] Xiangrui Meng commented on SPARK-17822: --- I will take a look. > JVMObjectTracker.ob

[jira] [Commented] (SPARK-18203) Spark Streaming Kinesis Extension v2.0.1 calls non-existent method

2016-12-02 Thread Nathan Canning (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716685#comment-15716685 ] Nathan Canning commented on SPARK-18203: It's not a compile time issue, it's a No

[jira] [Comment Edited] (SPARK-12717) pyspark broadcast fails when using multiple threads

2016-12-02 Thread Alexey Gritsenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716576#comment-15716576 ] Alexey Gritsenko edited comment on SPARK-12717 at 12/2/16 9:31 PM:

[jira] [Commented] (SPARK-12717) pyspark broadcast fails when using multiple threads

2016-12-02 Thread Alexey Gritsenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716576#comment-15716576 ] Alexey Gritsenko commented on SPARK-12717: -- I'm having the same problem when try

[jira] [Resolved] (SPARK-18670) Limit the number of StreamingQueryListener.StreamProgressEvent when there is no data

2016-12-02 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-18670. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 16108 [https://g

[jira] [Updated] (SPARK-18689) Support prioritized apps utilizing linux cgroups

2016-12-02 Thread Travis Hegner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Hegner updated SPARK-18689: -- Description: I'd like to propose a new mode of operation for the app scheduling system in a st

[jira] [Updated] (SPARK-18689) Support prioritized apps utilizing linux cgroups

2016-12-02 Thread Travis Hegner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Hegner updated SPARK-18689: -- Description: I'd like to propose a new mode of operation for the app scheduling system in a st

[jira] [Updated] (SPARK-18689) Support prioritized apps utilizing linux cgroups

2016-12-02 Thread Travis Hegner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Hegner updated SPARK-18689: -- Description: I'd like to propose a new mode of operation for the app scheduling system in a st

[jira] [Assigned] (SPARK-18690) Backward compatibility of unbounded frames

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18690: Assignee: Apache Spark > Backward compatibility of unbounded frames >

[jira] [Assigned] (SPARK-18690) Backward compatibility of unbounded frames

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18690: Assignee: (was: Apache Spark) > Backward compatibility of unbounded frames > -

[jira] [Updated] (SPARK-16554) Spark should kill executors when they are blacklisted

2016-12-02 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-16554: - Assignee: Jose Soltren > Spark should kill executors when they are blacklisted >

[jira] [Commented] (SPARK-18690) Backward compatibility of unbounded frames

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716182#comment-15716182 ] Apache Spark commented on SPARK-18690: -- User 'zero323' has created a pull request fo

[jira] [Updated] (SPARK-16654) UI Should show blacklisted executors & nodes

2016-12-02 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-16654: - Assignee: Jose Soltren > UI Should show blacklisted executors & nodes > -

[jira] [Created] (SPARK-18690) Backward compatibility of unbounded frames

2016-12-02 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-18690: -- Summary: Backward compatibility of unbounded frames Key: SPARK-18690 URL: https://issues.apache.org/jira/browse/SPARK-18690 Project: Spark Issue

[jira] [Updated] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-12-02 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-17636: --- Affects Version/s: 2.0.2 > Parquet filter push down doesn't handle struct fields > --

[jira] [Created] (SPARK-18689) Support prioritized apps utilizing linux cgroups

2016-12-02 Thread Travis Hegner (JIRA)
Travis Hegner created SPARK-18689: - Summary: Support prioritized apps utilizing linux cgroups Key: SPARK-18689 URL: https://issues.apache.org/jira/browse/SPARK-18689 Project: Spark Issue Type

[jira] [Commented] (SPARK-15573) Backwards-compatible persistence for spark.ml

2016-12-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715988#comment-15715988 ] Joseph K. Bradley commented on SPARK-15573: --- I've started a design doc and prot

[jira] [Assigned] (SPARK-15573) Backwards-compatible persistence for spark.ml

2016-12-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-15573: - Assignee: Joseph K. Bradley > Backwards-compatible persistence for spark.ml > --

[jira] [Closed] (SPARK-11705) Eliminate unnecessary Cartesian Join

2016-12-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-11705. --- Resolution: Cannot Reproduce > Eliminate unnecessary Cartesian Join > ---

[jira] [Assigned] (SPARK-18681) Throw Filtering is supported only on partition keys of type string exception

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18681: Assignee: Apache Spark > Throw Filtering is supported only on partition keys of type strin

[jira] [Assigned] (SPARK-18681) Throw Filtering is supported only on partition keys of type string exception

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18681: Assignee: (was: Apache Spark) > Throw Filtering is supported only on partition keys of

[jira] [Commented] (SPARK-18681) Throw Filtering is supported only on partition keys of type string exception

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715860#comment-15715860 ] Apache Spark commented on SPARK-18681: -- User 'wangyum' has created a pull request fo

[jira] [Commented] (SPARK-15815) Hang while enable blacklistExecutor and DynamicExecutorAllocator

2016-12-02 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715719#comment-15715719 ] Imran Rashid commented on SPARK-15815: -- [~SuYan] I've been mulling this over for a w

[jira] [Commented] (SPARK-18640) Fix minor synchronization issue in TaskSchedulerImpl.runningTasksByExecutors

2016-12-02 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715712#comment-15715712 ] Josh Rosen commented on SPARK-18640: Actually, this doesn't look necessary because th

[jira] [Commented] (SPARK-18640) Fix minor synchronization issue in TaskSchedulerImpl.runningTasksByExecutors

2016-12-02 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715690#comment-15715690 ] Josh Rosen commented on SPARK-18640: I'm also going to backport this into branch-1.6.

[jira] [Commented] (SPARK-11705) Eliminate unnecessary Cartesian Join

2016-12-02 Thread Andrew Ray (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715675#comment-15715675 ] Andrew Ray commented on SPARK-11705: Above example does not have a cartesian product

[jira] [Updated] (SPARK-18681) Throw Filtering is supported only on partition keys of type string exception

2016-12-02 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-18681: Description: I'm using MySQL for the Hive Metastore. {{hive.metastore.try.direct.sql=true}} and {{

[jira] [Resolved] (SPARK-18677) Json path implementation fails to parse ['key']

2016-12-02 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18677. --- Resolution: Fixed Assignee: Ryan Blue Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-18640) Fix minor synchronization issue in TaskSchedulerImpl.runningTasksByExecutors

2016-12-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715503#comment-15715503 ] Andrew Or commented on SPARK-18640: --- my mistake > Fix minor synchronization issue in T

[jira] [Commented] (SPARK-16589) Chained cartesian produces incorrect number of records

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715474#comment-15715474 ] Apache Spark commented on SPARK-16589: -- User 'aray' has created a pull request for t

[jira] [Assigned] (SPARK-18634) Corruption and Correctness issues with exploding Python UDFs

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18634: Assignee: (was: Apache Spark) > Corruption and Correctness issues with exploding Pytho

[jira] [Assigned] (SPARK-18634) Corruption and Correctness issues with exploding Python UDFs

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18634: Assignee: Apache Spark > Corruption and Correctness issues with exploding Python UDFs > --

[jira] [Commented] (SPARK-18634) Corruption and Correctness issues with exploding Python UDFs

2016-12-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715468#comment-15715468 ] Apache Spark commented on SPARK-18634: -- User 'viirya' has created a pull request for

[jira] [Commented] (SPARK-17945) Writing to S3 should allow setting object metadata

2016-12-02 Thread Jeff Schobelock (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715396#comment-15715396 ] Jeff Schobelock commented on SPARK-17945: - Just wanted to add some more justifica

[jira] [Commented] (SPARK-18688) Interpolated time series join

2016-12-02 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715201#comment-15715201 ] Herman van Hovell commented on SPARK-18688: --- For my reference: Could you give a

[jira] [Resolved] (SPARK-18659) Incorrect behaviors in overwrite table for datasource tables

2016-12-02 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18659. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.1.0 > Incorrect behaviors

[jira] [Resolved] (SPARK-18419) `JDBCRelation.insert` should not remove Spark options

2016-12-02 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18419. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.1.0 > `JDBCRelation.in

[jira] [Commented] (SPARK-18678) Skewed feature subsampling in Random forest

2016-12-02 Thread Bjoern Toldbod (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715169#comment-15715169 ] Bjoern Toldbod commented on SPARK-18678: I'm not familiar with this subsampling t

[jira] [Resolved] (SPARK-18679) Regression in file listing performance

2016-12-02 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18679. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.1.0 > Regression in file

[jira] [Created] (SPARK-18688) Interpolated time series join

2016-12-02 Thread Jarno Seppanen (JIRA)
Jarno Seppanen created SPARK-18688: -- Summary: Interpolated time series join Key: SPARK-18688 URL: https://issues.apache.org/jira/browse/SPARK-18688 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2016-12-02 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714968#comment-15714968 ] Herman van Hovell edited comment on SPARK-18609 at 12/2/16 12:13 PM: --

[jira] [Commented] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2016-12-02 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714968#comment-15714968 ] Herman van Hovell commented on SPARK-18609: --- Those are two separate problems, t

[jira] [Updated] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2016-12-02 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18609: -- Target Version/s: 2.1.1 (was: 2.2.0) > [SQL] column mixup with CROSS JOIN > --

[jira] [Updated] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2016-12-02 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18609: -- Target Version/s: 2.2.0 > [SQL] column mixup with CROSS JOIN >

[jira] [Commented] (SPARK-18678) Skewed feature subsampling in Random forest

2016-12-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714960#comment-15714960 ] Sean Owen commented on SPARK-18678: --- It's helpful if you point out specifically what yo

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2016-12-02 Thread Robert Neumann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714957#comment-15714957 ] Robert Neumann commented on SPARK-650: -- OK. Will do. > Add a "setup hook" API for run

[jira] [Commented] (SPARK-18684) Spark Executors off-heap memory usage keeps increasing while running spark streaming

2016-12-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714933#comment-15714933 ] Sean Owen commented on SPARK-18684: --- That is not, in general, a bug. You need more off-

[jira] [Updated] (SPARK-18629) Fix numPartition of JDBCSuite Testcase

2016-12-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18629: -- Assignee: Weiqing Yang Priority: Trivial (was: Minor) > Fix numPartition of JDBCSuite Testcase > -

[jira] [Resolved] (SPARK-18629) Fix numPartition of JDBCSuite Testcase

2016-12-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18629. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16062 [https://github.co

[jira] [Commented] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2016-12-02 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714899#comment-15714899 ] Furcy Pin commented on SPARK-18609: --- Hi, I found another minimal example that yields a

[jira] [Commented] (SPARK-17557) SQL query on parquet table java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary

2016-12-02 Thread Jarno Seppanen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714820#comment-15714820 ] Jarno Seppanen commented on SPARK-17557: Got the same error on pyspark on spark 2

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2016-12-02 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714819#comment-15714819 ] Herman van Hovell commented on SPARK-650: - [~lars_francke][~Skamandros][~rneumann]

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2016-12-02 Thread Robert Neumann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714762#comment-15714762 ] Robert Neumann commented on SPARK-650: -- Sean, I agree this is the essential question i

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2016-12-02 Thread Robert Neumann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714760#comment-15714760 ] Robert Neumann commented on SPARK-650: -- Sean, I agree this is the essential question i

[jira] [Issue Comment Deleted] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2016-12-02 Thread Robert Neumann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Neumann updated SPARK-650: - Comment: was deleted (was: Sean, I agree this is the essential question in this thread. If we get

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2016-12-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15714725#comment-15714725 ] Sean Owen commented on SPARK-650: - Why would a singleton not work? This is really the essen

  1   2   >