[jira] [Updated] (SPARK-15799) Release SparkR on CRAN

2016-06-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15799: -- Target Version/s: 2.1.0 > Release SparkR on CRAN > -- > > K

[jira] [Updated] (SPARK-15581) MLlib 2.1 Roadmap

2016-06-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15581: -- Description: This is a master list for MLlib improvements we are working on for the next relea

[jira] [Created] (SPARK-15799) Release SparkR on CRAN

2016-06-06 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-15799: - Summary: Release SparkR on CRAN Key: SPARK-15799 URL: https://issues.apache.org/jira/browse/SPARK-15799 Project: Spark Issue Type: New Feature Co

[jira] [Updated] (SPARK-15581) MLlib 2.1 Roadmap

2016-06-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15581: -- Description: This is a master list for MLlib improvements we are working on for the next relea

[jira] [Commented] (SPARK-3451) spark-submit should support specifying glob wildcards in the --jars CLI option

2016-06-06 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317975#comment-15317975 ] Jeff Zhang commented on SPARK-3451: --- +1 for this feature, or allow specifying jar folder

[jira] [Comment Edited] (SPARK-15779) SQL context fails when Hive uses Tez as its default execution engine

2016-06-06 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317959#comment-15317959 ] Jeff Zhang edited comment on SPARK-15779 at 6/7/16 6:27 AM: A

[jira] [Commented] (SPARK-15779) SQL context fails when Hive uses Tez as its default execution engine

2016-06-06 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317959#comment-15317959 ] Jeff Zhang commented on SPARK-15779: You need to specify hive.execution.engine=mr in

[jira] [Resolved] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

2016-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15792. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13535 [https://github.

[jira] [Resolved] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15632. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13529 [https://github.

[jira] [Created] (SPARK-15798) Secondary sort in Dataset/DataFrame

2016-06-06 Thread koert kuipers (JIRA)
koert kuipers created SPARK-15798: - Summary: Secondary sort in Dataset/DataFrame Key: SPARK-15798 URL: https://issues.apache.org/jira/browse/SPARK-15798 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-14525) DataFrameWriter's save method should delegate to jdbc for jdbc datasource

2016-06-06 Thread Justin Pihony (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317846#comment-15317846 ] Justin Pihony commented on SPARK-14525: --- [~rxin] I have pushed my changes that now

[jira] [Commented] (SPARK-15797) To expose groupingSets for DataFrame

2016-06-06 Thread Priyanka Garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317829#comment-15317829 ] Priyanka Garg commented on SPARK-15797: --- I am working on this. > To expose groupin

[jira] [Created] (SPARK-15797) To expose groupingSets for DataFrame

2016-06-06 Thread Priyanka Garg (JIRA)
Priyanka Garg created SPARK-15797: - Summary: To expose groupingSets for DataFrame Key: SPARK-15797 URL: https://issues.apache.org/jira/browse/SPARK-15797 Project: Spark Issue Type: New Featur

[jira] [Commented] (SPARK-9221) Support IntervalType in Range Frame

2016-06-06 Thread Yang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317813#comment-15317813 ] Yang Wang commented on SPARK-9221: -- Any progress on this? [~hvanhovell] > Support Interv

[jira] [Commented] (SPARK-15755) java.lang.NullPointerException when run spark 2.0 setting spark.serializer=org.apache.spark.serializer.KryoSerializer

2016-06-06 Thread marymwu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317782#comment-15317782 ] marymwu commented on SPARK-15755: - Any comments? > java.lang.NullPointerException when r

[jira] [Commented] (SPARK-15757) Error occurs when using Spark sql "select" statement on orc file after hive sql "insert overwrite tb1 select * from sourcTb" has been executed on this orc file

2016-06-06 Thread marymwu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317778#comment-15317778 ] marymwu commented on SPARK-15757: - The error occurs steps are as follows: hope it helps 1

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to slo

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to slo

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to slo

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to slo

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to slo

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to slo

[jira] [Created] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
Gabor Feher created SPARK-15796: --- Summary: Spark 1.6 default memory settings can cause heavy GC when caching Key: SPARK-15796 URL: https://issues.apache.org/jira/browse/SPARK-15796 Project: Spark

[jira] [Updated] (SPARK-14485) Task finished cause fetch failure when its executor has already been removed by driver

2016-06-06 Thread iward (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] iward updated SPARK-14485: -- Summary: Task finished cause fetch failure when its executor has already been removed by driver (was: Task fi

[jira] [Assigned] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15795: Assignee: Apache Spark > Enable more optimizations in whole stage codegen when isNull is a

[jira] [Assigned] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15795: Assignee: (was: Apache Spark) > Enable more optimizations in whole stage codegen when

[jira] [Commented] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317707#comment-15317707 ] Apache Spark commented on SPARK-15795: -- User 'inouehrs' has created a pull request f

[jira] [Created] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

2016-06-06 Thread Hiroshi Inoue (JIRA)
Hiroshi Inoue created SPARK-15795: - Summary: Enable more optimizations in whole stage codegen when isNull is a compile-time constant Key: SPARK-15795 URL: https://issues.apache.org/jira/browse/SPARK-15795

[jira] [Assigned] (SPARK-15794) Should truncate toString() of very wide schemas

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15794: Assignee: (was: Apache Spark) > Should truncate toString() of very wide schemas >

[jira] [Assigned] (SPARK-15794) Should truncate toString() of very wide schemas

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15794: Assignee: Apache Spark > Should truncate toString() of very wide schemas > ---

[jira] [Commented] (SPARK-15794) Should truncate toString() of very wide schemas

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317651#comment-15317651 ] Apache Spark commented on SPARK-15794: -- User 'ericl' has created a pull request for

[jira] [Created] (SPARK-15794) Should truncate toString() of very wide schemas

2016-06-06 Thread Eric Liang (JIRA)
Eric Liang created SPARK-15794: -- Summary: Should truncate toString() of very wide schemas Key: SPARK-15794 URL: https://issues.apache.org/jira/browse/SPARK-15794 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-9876) Upgrade parquet-mr to 1.8.1

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9876: --- Assignee: Ryan Blue (was: Apache Spark) > Upgrade parquet-mr to 1.8.1 >

[jira] [Assigned] (SPARK-9876) Upgrade parquet-mr to 1.8.1

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9876: --- Assignee: Apache Spark (was: Ryan Blue) > Upgrade parquet-mr to 1.8.1 >

[jira] [Commented] (SPARK-12655) GraphX does not unpersist RDDs

2016-06-06 Thread Alexander Pivovarov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317577#comment-15317577 ] Alexander Pivovarov commented on SPARK-12655: - [~srowen] Can the fix be added

[jira] [Commented] (SPARK-15574) Python meta-algorithms in Scala

2016-06-06 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317503#comment-15317503 ] Xusen Yin commented on SPARK-15574: --- [~josephkb] Can I work on this one? > Python met

[jira] [Assigned] (SPARK-15793) Word2vec in ML package should have maxSentenceLength method

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15793: Assignee: Apache Spark > Word2vec in ML package should have maxSentenceLength method > ---

[jira] [Commented] (SPARK-15793) Word2vec in ML package should have maxSentenceLength method

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317487#comment-15317487 ] Apache Spark commented on SPARK-15793: -- User 'yinxusen' has created a pull request f

[jira] [Assigned] (SPARK-15793) Word2vec in ML package should have maxSentenceLength method

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15793: Assignee: (was: Apache Spark) > Word2vec in ML package should have maxSentenceLength m

[jira] [Commented] (SPARK-14381) Review spark.ml parity for feature transformers

2016-06-06 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317459#comment-15317459 ] Xusen Yin commented on SPARK-14381: --- Comparing mllib.feature with ml.feature, there are

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317460#comment-15317460 ] Cheng Lian commented on SPARK-15632: The {{.map(identity)}} example is quite interest

[jira] [Created] (SPARK-15793) Word2vec in ML package should have maxSentenceLength method

2016-06-06 Thread Xusen Yin (JIRA)
Xusen Yin created SPARK-15793: - Summary: Word2vec in ML package should have maxSentenceLength method Key: SPARK-15793 URL: https://issues.apache.org/jira/browse/SPARK-15793 Project: Spark Issue

[jira] [Resolved] (SPARK-15652) Missing org.apache.spark.launcher.SparkAppHandle.Listener notification if SparkSubmit JVM shutsdown

2016-06-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-15652. Resolution: Fixed Assignee: Subroto Sanyal Fix Version/s: 2.0.0 > Missing o

[jira] [Updated] (SPARK-15791) NPE in ScalarSubquery

2016-06-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15791: --- Assignee: Eric Liang (was: Davies Liu) > NPE in ScalarSubquery > - > >

[jira] [Commented] (SPARK-14146) Imported implicits can't be found in Spark REPL in some cases

2016-06-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317383#comment-15317383 ] Shixiong Zhu commented on SPARK-14146: -- [~scrapco...@gmail.com] could you submit a P

[jira] [Assigned] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15792: Assignee: Apache Spark > [SQL] Allows operator to change the verbosity in explain output.

[jira] [Commented] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317379#comment-15317379 ] Apache Spark commented on SPARK-15792: -- User 'clockfly' has created a pull request f

[jira] [Assigned] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15792: Assignee: (was: Apache Spark) > [SQL] Allows operator to change the verbosity in expla

[jira] [Created] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

2016-06-06 Thread Sean Zhong (JIRA)
Sean Zhong created SPARK-15792: -- Summary: [SQL] Allows operator to change the verbosity in explain output. Key: SPARK-15792 URL: https://issues.apache.org/jira/browse/SPARK-15792 Project: Spark

[jira] [Created] (SPARK-15791) NPE in ScalarSubquery

2016-06-06 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15791: -- Summary: NPE in ScalarSubquery Key: SPARK-15791 URL: https://issues.apache.org/jira/browse/SPARK-15791 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-15716) Memory usage of driver keeps growing up in Spark Streaming

2016-06-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317340#comment-15317340 ] Shixiong Zhu commented on SPARK-15716: -- [~yani.chen] Are you using the Apache Spark

[jira] [Assigned] (SPARK-15789) Allow reserved keywords in most places

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15789: Assignee: (was: Apache Spark) > Allow reserved keywords in most places > -

[jira] [Commented] (SPARK-15789) Allow reserved keywords in most places

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317320#comment-15317320 ] Apache Spark commented on SPARK-15789: -- User 'hvanhovell' has created a pull request

[jira] [Assigned] (SPARK-15789) Allow reserved keywords in most places

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15789: Assignee: Apache Spark > Allow reserved keywords in most places >

[jira] [Assigned] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15781: Assignee: Apache Spark > Misleading deprecated property in standalone cluster configuratio

[jira] [Assigned] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15781: Assignee: (was: Apache Spark) > Misleading deprecated property in standalone cluster c

[jira] [Commented] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317241#comment-15317241 ] Apache Spark commented on SPARK-15781: -- User 'bomeng' has created a pull request for

[jira] [Updated] (SPARK-15782) --packages doesn't work with the spark-shell

2016-06-06 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nezih Yigitbasi updated SPARK-15782: Summary: --packages doesn't work with the spark-shell (was: --packages doesn't work the sp

[jira] [Created] (SPARK-15790) Audit @Since annotations in ML

2016-06-06 Thread Nick Pentreath (JIRA)
Nick Pentreath created SPARK-15790: -- Summary: Audit @Since annotations in ML Key: SPARK-15790 URL: https://issues.apache.org/jira/browse/SPARK-15790 Project: Spark Issue Type: Documentation

[jira] [Created] (SPARK-15788) PySpark IDFModel missing "idf" property

2016-06-06 Thread Nick Pentreath (JIRA)
Nick Pentreath created SPARK-15788: -- Summary: PySpark IDFModel missing "idf" property Key: SPARK-15788 URL: https://issues.apache.org/jira/browse/SPARK-15788 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-15789) Allow reserved keywords in most places

2016-06-06 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-15789: - Summary: Allow reserved keywords in most places Key: SPARK-15789 URL: https://issues.apache.org/jira/browse/SPARK-15789 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15778. --- Resolution: Fixed > Add 2.0.0-preview to dropdown / reorg description of previews at > spark.apache.

[jira] [Assigned] (SPARK-15204) Improve nullability inference for Aggregator

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15204: Assignee: (was: Apache Spark) > Improve nullability inference for Aggregator > ---

[jira] [Commented] (SPARK-15204) Improve nullability inference for Aggregator

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317187#comment-15317187 ] Apache Spark commented on SPARK-15204: -- User 'koertkuipers' has created a pull reque

[jira] [Assigned] (SPARK-15204) Improve nullability inference for Aggregator

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15204: Assignee: Apache Spark > Improve nullability inference for Aggregator > --

[jira] [Commented] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317178#comment-15317178 ] Apache Spark commented on SPARK-15654: -- User 'davies' has created a pull request for

[jira] [Commented] (SPARK-7960) Serialization problem when multiple receivers are specified in a loop

2016-06-06 Thread UTKARSH BHATNAGAR (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317176#comment-15317176 ] UTKARSH BHATNAGAR commented on SPARK-7960: -- DEBUG Output: 16/06/06 20:29:00 DEBU

[jira] [Reopened] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-15778: --- One more comment from [~rxin] -- maybe put the preview releases in a separate section in the dropdown. L

[jira] [Updated] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15778: -- Attachment: Screen Shot 2016-06-06 at 21.25.59.png Now with Stable/Preview section. It's sort of funny

[jira] [Commented] (SPARK-15507) ClassCastException: SomeCaseClass cannot be cast to org.apache.spark.sql.Row

2016-06-06 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317156#comment-15317156 ] koert kuipers commented on SPARK-15507: --- also soon we will not try to do this in a

[jira] [Commented] (SPARK-15783) Fix more flakiness: o.a.s.scheduler.BlacklistIntegrationSuite

2016-06-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317155#comment-15317155 ] Imran Rashid commented on SPARK-15783: -- Turned the tests off via https://github.com/

[jira] [Commented] (SPARK-15069) GSoC 2016: Exposing more R and Python APIs for MLlib

2016-06-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317151#comment-15317151 ] Joseph K. Bradley commented on SPARK-15069: --- h4. 6/6/2016 - Week 3 To-do items

[jira] [Updated] (SPARK-15767) Decision Tree Regression wrapper in SparkR

2016-06-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15767: -- Assignee: Kai Jiang Affects Version/s: (was: 2.1.0) Target Version

[jira] [Commented] (SPARK-15672) R programming guide update

2016-06-06 Thread Kai Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317133#comment-15317133 ] Kai Jiang commented on SPARK-15672: --- Working on it. > R programming guide update > ---

[jira] [Commented] (SPARK-15507) ClassCastException: SomeCaseClass cannot be cast to org.apache.spark.sql.Row

2016-06-06 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317131#comment-15317131 ] koert kuipers commented on SPARK-15507: --- we do not know all the columns, but we do

[jira] [Commented] (SPARK-7960) Serialization problem when multiple receivers are specified in a loop

2016-06-06 Thread UTKARSH BHATNAGAR (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317103#comment-15317103 ] UTKARSH BHATNAGAR commented on SPARK-7960: -- This problem persists in Spark 1.6.1

[jira] [Resolved] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15778. --- Resolution: Fixed Fix Version/s: 2.0.0 Live at http://spark.apache.org/downloads.html > Add 2

[jira] [Commented] (SPARK-15507) ClassCastException: SomeCaseClass cannot be cast to org.apache.spark.sql.Row

2016-06-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317084#comment-15317084 ] Wenchen Fan commented on SPARK-15507: - if you have a RDD[Row], just call `rdd.toDF().

[jira] [Assigned] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15654: -- Assignee: Davies Liu (was: Takeshi Yamamuro) > Reading gzipped files results in duplicate row

[jira] [Commented] (SPARK-14279) Improve the spark build to pick the version information from the pom file and add git commit information

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317005#comment-15317005 ] Apache Spark commented on SPARK-14279: -- User 'dhruve' has created a pull request for

[jira] [Updated] (SPARK-15786) joinWith bytecode generation calling ByteBuffer.wrap with InternalRow

2016-06-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-15786: - Target Version/s: 2.0.0 > joinWith bytecode generation calling ByteBuffer.wrap with Inter

[jira] [Created] (SPARK-15787) Display more helpful error messages for several invalid operations

2016-06-06 Thread nalin garg (JIRA)
nalin garg created SPARK-15787: -- Summary: Display more helpful error messages for several invalid operations Key: SPARK-15787 URL: https://issues.apache.org/jira/browse/SPARK-15787 Project: Spark

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316998#comment-15316998 ] Sean Zhong commented on SPARK-15632: *Root cause analysis:* *The root cause is that

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316996#comment-15316996 ] Sean Zhong commented on SPARK-15632: There are more issues linked with this bug which

[jira] [Created] (SPARK-15786) Spark SQL - joinWith bytecode generation calling ByteBuffer.wrap with InternalRow

2016-06-06 Thread Richard Marscher (JIRA)
Richard Marscher created SPARK-15786: Summary: Spark SQL - joinWith bytecode generation calling ByteBuffer.wrap with InternalRow Key: SPARK-15786 URL: https://issues.apache.org/jira/browse/SPARK-15786

[jira] [Updated] (SPARK-15786) joinWith bytecode generation calling ByteBuffer.wrap with InternalRow

2016-06-06 Thread Richard Marscher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Marscher updated SPARK-15786: - Summary: joinWith bytecode generation calling ByteBuffer.wrap with InternalRow (was: Spa

[jira] [Assigned] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15632: Assignee: Xiang Zhong (was: Apache Spark) > Dataset typed filter operation changes query

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316979#comment-15316979 ] Apache Spark commented on SPARK-15632: -- User 'clockfly' has created a pull request f

[jira] [Assigned] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15632: Assignee: Apache Spark (was: Xiang Zhong) > Dataset typed filter operation changes query

[jira] [Resolved] (SPARK-15764) Replace n^2 loop in BindReferences

2016-06-06 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-15764. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13505 [https://github.

[jira] [Updated] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15654: --- Assignee: Takeshi Yamamuro > Reading gzipped files results in duplicate rows > --

[jira] [Commented] (SPARK-15780) Support mapValues on KeyValueGroupedDataset

2016-06-06 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316943#comment-15316943 ] koert kuipers commented on SPARK-15780: --- original discussion is here: https://mail-

[jira] [Commented] (SPARK-11159) Nested SQL UDF raises java.lang.UnsupportedOperationException: Cannot evaluate expression

2016-06-06 Thread Christoph Tavan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316922#comment-15316922 ] Christoph Tavan commented on SPARK-11159: - I can confirm this bug with spark 1.6.

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

2016-06-06 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316908#comment-15316908 ] Shivaram Venkataraman commented on SPARK-13525: --- [~vmenda] Your stack trace

[jira] [Comment Edited] (SPARK-14767) Codegen "no constructor found" errors with Maps inside case classes in Datasets

2016-06-06 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316904#comment-15316904 ] Burak Yavuz edited comment on SPARK-14767 at 6/6/16 6:11 PM: -

[jira] [Commented] (SPARK-14767) Codegen "no constructor found" errors with Maps inside case classes in Datasets

2016-06-06 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316904#comment-15316904 ] Burak Yavuz commented on SPARK-14767: - I still run into this > Codegen "no construct

[jira] [Commented] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316899#comment-15316899 ] Reynold Xin commented on SPARK-15778: - See https://issues.apache.org/jira/browse/INFR

[jira] [Commented] (SPARK-15159) SparkSession R API

2016-06-06 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316898#comment-15316898 ] Shivaram Venkataraman commented on SPARK-15159: --- That sounds good. Also one

[jira] [Updated] (SPARK-8426) Add blacklist mechanism for YARN container allocation

2016-06-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-8426: Attachment: DesignDocforBlacklistMechanism.pdf Updated Design Doc, based on performance tests and an

[jira] [Comment Edited] (SPARK-15687) Columnar execution engine

2016-06-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316858#comment-15316858 ] Kazuaki Ishizaki edited comment on SPARK-15687 at 6/6/16 5:49 PM: -

  1   2   >