[jira] [Updated] (SPARK-21168) KafkaRDD should always set kafka clientId.

[ https://issues.apache.org/jira/browse/SPARK-21168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21168: -- Target Version/s: (was: 2.0.2) > KafkaRDD should always set kafka clientId. > ---

[jira] [Updated] (SPARK-18267) Distribute PySpark via Python Package Index (pypi)

[ https://issues.apache.org/jira/browse/SPARK-18267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18267: -- Target Version/s: (was: 2.2.0) > Distribute PySpark via Python Package Index (pypi) > ---

[jira] [Created] (SPARK-21206) the window slice of Dstream is wrong

Fei Shao created SPARK-21206: Summary: the window slice of Dstream is wrong Key: SPARK-21206 URL: https://issues.apache.org/jira/browse/SPARK-21206 Project: Spark Issue Type: Bug Compon

[jira] [Commented] (SPARK-21206) the window slice of Dstream is wrong

[ https://issues.apache.org/jira/browse/SPARK-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062276#comment-16062276 ] Sean Owen commented on SPARK-21206: --- I'm not clear what you are reporting here. What's

[jira] [Commented] (SPARK-21204) RuntimeException with Set and Case Class in Spark 2.1.1

[ https://issues.apache.org/jira/browse/SPARK-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062277#comment-16062277 ] Takeshi Yamamuro commented on SPARK-21204: -- I think the query is invalid: {code}

[jira] [Created] (SPARK-21207) ML/MLLIB Save Word2Vec Yarn Cluster

offvolt created SPARK-21207: --- Summary: ML/MLLIB Save Word2Vec Yarn Cluster Key: SPARK-21207 URL: https://issues.apache.org/jira/browse/SPARK-21207 Project: Spark Issue Type: Question Com

[jira] [Assigned] (SPARK-21204) RuntimeException with Set and Case Class in Spark 2.1.1

[ https://issues.apache.org/jira/browse/SPARK-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21204: Assignee: (was: Apache Spark) > RuntimeException with Set and Case Class in Spark 2.1.

[jira] [Commented] (SPARK-21204) RuntimeException with Set and Case Class in Spark 2.1.1

[ https://issues.apache.org/jira/browse/SPARK-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062283#comment-16062283 ] Apache Spark commented on SPARK-21204: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-21204) RuntimeException with Set and Case Class in Spark 2.1.1

[ https://issues.apache.org/jira/browse/SPARK-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21204: Assignee: Apache Spark > RuntimeException with Set and Case Class in Spark 2.1.1 > ---

[jira] [Commented] (SPARK-21206) the window slice of Dstream is wrong

[ https://issues.apache.org/jira/browse/SPARK-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062284#comment-16062284 ] Fei Shao commented on SPARK-21206: -- Hi Sean Owen, I am sorry, I did not give enough me

[jira] [Comment Edited] (SPARK-21206) the window slice of Dstream is wrong

[ https://issues.apache.org/jira/browse/SPARK-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062284#comment-16062284 ] Fei Shao edited comment on SPARK-21206 at 6/25/17 10:57 AM: H

[jira] [Resolved] (SPARK-16649) Push partition predicates down into metastore for OptimizeMetadataOnlyQuery

[ https://issues.apache.org/jira/browse/SPARK-16649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16649. -- Resolution: Cannot Reproduce I am resolving this per https://github.com/apache/spark/pull/1428

[jira] [Resolved] (SPARK-16946) saveAsTable[append] with different number of columns should throw Exception

[ https://issues.apache.org/jira/browse/SPARK-16946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16946. -- Resolution: Cannot Reproduce I am resolving this per https://github.com/apache/spark/pull/1453

[jira] [Resolved] (SPARK-21207) ML/MLLIB Save Word2Vec Yarn Cluster

[ https://issues.apache.org/jira/browse/SPARK-21207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21207. --- Resolution: Invalid Target Version/s: (was: 2.0.1) Questions go on the mailing list, like

[jira] [Resolved] (SPARK-11057) SQL: corr and cov for many columns

[ https://issues.apache.org/jira/browse/SPARK-11057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11057. --- Resolution: Won't Fix > SQL: corr and cov for many columns > -- > >

[jira] [Resolved] (SPARK-10796) The Stage taskSets may are all removed while stage still have pending partitions after having lost some executors

[ https://issues.apache.org/jira/browse/SPARK-10796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10796. --- Resolution: Won't Fix > The Stage taskSets may are all removed while stage still have pending > part

[jira] [Resolved] (SPARK-12948) Consider reducing size of broadcasts in OrcRelation

[ https://issues.apache.org/jira/browse/SPARK-12948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12948. --- Resolution: Won't Fix > Consider reducing size of broadcasts in OrcRelation > ---

[jira] [Resolved] (SPARK-15763) Add DELETE FILE command support in spark

[ https://issues.apache.org/jira/browse/SPARK-15763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15763. --- Resolution: Won't Fix > Add DELETE FILE command support in spark > --

[jira] [Resolved] (SPARK-19869) move table related ddl from ddl.scala to tables.scala

[ https://issues.apache.org/jira/browse/SPARK-19869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19869. --- Resolution: Won't Fix > move table related ddl from ddl.scala to tables.scala > -

[jira] [Resolved] (SPARK-20413) New Optimizer Hint to prevent collapsing of adjacent projections

[ https://issues.apache.org/jira/browse/SPARK-20413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20413. --- Resolution: Won't Fix > New Optimizer Hint to prevent collapsing of adjacent projections > --

[jira] [Resolved] (SPARK-20115) Fix DAGScheduler to recompute all the lost shuffle blocks when external shuffle service is unavailable

[ https://issues.apache.org/jira/browse/SPARK-20115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20115. --- Resolution: Won't Fix > Fix DAGScheduler to recompute all the lost shuffle blocks when external > sh

[jira] [Created] (SPARK-21208) Ability to "setLocalProperty" from sc, in sparkR

Karuppayya created SPARK-21208: -- Summary: Ability to "setLocalProperty" from sc, in sparkR Key: SPARK-21208 URL: https://issues.apache.org/jira/browse/SPARK-21208 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062368#comment-16062368 ] Yuming Wang commented on SPARK-21067: - Are you enable [HDFS Federation|http://hadoop

[jira] [Commented] (SPARK-21063) Spark return an empty result from remote hadoop cluster

[ https://issues.apache.org/jira/browse/SPARK-21063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062370#comment-16062370 ] Yuming Wang commented on SPARK-21063: - May be you should do this before submit your S

[jira] [Commented] (SPARK-21208) Ability to "setLocalProperty" from sc, in sparkR

[ https://issues.apache.org/jira/browse/SPARK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062396#comment-16062396 ] Felix Cheung commented on SPARK-21208: -- +1 > Ability to "setLocalProperty" from sc,

[jira] [Resolved] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

[ https://issues.apache.org/jira/browse/SPARK-21093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-21093. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.3.0 Targe

[jira] [Commented] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

[ https://issues.apache.org/jira/browse/SPARK-21093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062399#comment-16062399 ] Felix Cheung commented on SPARK-21093: -- since this is a very core change to SparkR,

[jira] [Comment Edited] (SPARK-12806) Support SQL expressions extracting values from VectorUDT

[ https://issues.apache.org/jira/browse/SPARK-12806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061718#comment-16061718 ] Franklyn Dsouza edited comment on SPARK-12806 at 6/25/17 11:32 PM:

[jira] [Commented] (SPARK-21208) Ability to "setLocalProperty" from sc, in sparkR

[ https://issues.apache.org/jira/browse/SPARK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062460#comment-16062460 ] Hyukjin Kwon commented on SPARK-21208: -- [~karup1990], are you working on this? I sen

[jira] [Comment Edited] (SPARK-21208) Ability to "setLocalProperty" from sc, in sparkR

[ https://issues.apache.org/jira/browse/SPARK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062460#comment-16062460 ] Hyukjin Kwon edited comment on SPARK-21208 at 6/26/17 12:24 AM: ---

[jira] [Commented] (SPARK-21204) RuntimeException with Set and Case Class in Spark 2.1.1

[ https://issues.apache.org/jira/browse/SPARK-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062488#comment-16062488 ] Liang-Chi Hsieh commented on SPARK-21204: - [~maropu] I think It's more likely to

[jira] [Commented] (SPARK-21198) SparkSession catalog is terribly slow

[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062489#comment-16062489 ] Liang-Chi Hsieh commented on SPARK-21198: - [~revolucion09] Any update? Because i

[jira] [Commented] (SPARK-21204) RuntimeException with Set and Case Class in Spark 2.1.1

[ https://issues.apache.org/jira/browse/SPARK-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062496#comment-16062496 ] Takeshi Yamamuro commented on SPARK-21204: -- ok, I'll check your pr later. Thanks

[jira] [Commented] (SPARK-21198) SparkSession catalog is terribly slow

[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062498#comment-16062498 ] Saif Addin commented on SPARK-21198: [~viirya] I'll take a look again on Monday, but

[jira] [Commented] (SPARK-21198) SparkSession catalog is terribly slow

[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062499#comment-16062499 ] Liang-Chi Hsieh commented on SPARK-21198: - Thanks, [~revolucion09]. Can you measu

[jira] [Commented] (SPARK-21208) Ability to "setLocalProperty" from sc, in sparkR

[ https://issues.apache.org/jira/browse/SPARK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062523#comment-16062523 ] Karuppayya commented on SPARK-21208: [~hyukjin.kwon] I am currentlty not working on t

[jira] [Commented] (SPARK-21208) Ability to "setLocalProperty" from sc, in sparkR

[ https://issues.apache.org/jira/browse/SPARK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062525#comment-16062525 ] Hyukjin Kwon commented on SPARK-21208: -- Here - SPARK-21149. > Ability to "setLocalP

[jira] [Assigned] (SPARK-19866) Add local version of Word2Vec findSynonyms for spark.ml: Python API

[ https://issues.apache.org/jira/browse/SPARK-19866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang reassigned SPARK-19866: --- Assignee: Xin Ren > Add local version of Word2Vec findSynonyms for spark.ml: Python API > --

[jira] [Assigned] (SPARK-19104) CompileException with Map and Case Class in Spark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19104: Assignee: (was: Apache Spark) > CompileException with Map and Case Class in Spark 2.1

[jira] [Commented] (SPARK-19104) CompileException with Map and Case Class in Spark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062528#comment-16062528 ] Apache Spark commented on SPARK-19104: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-19104) CompileException with Map and Case Class in Spark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19104: Assignee: Apache Spark > CompileException with Map and Case Class in Spark 2.1.0 > --

[jira] [Commented] (SPARK-19104) CompileException with Map and Case Class in Spark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062529#comment-16062529 ] Liang-Chi Hsieh commented on SPARK-19104: - Just found this issue. I proposed a PR

[jira] [Comment Edited] (SPARK-19104) CompileException with Map and Case Class in Spark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062529#comment-16062529 ] Liang-Chi Hsieh edited comment on SPARK-19104 at 6/26/17 4:37 AM: -

[jira] [Commented] (SPARK-20213) DataFrameWriter operations do not show up in SQL tab

[ https://issues.apache.org/jira/browse/SPARK-20213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062534#comment-16062534 ] Apache Spark commented on SPARK-20213: -- User 'cloud-fan' has created a pull request