[jira] [Resolved] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15632. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13529

[jira] [Created] (SPARK-15798) Secondary sort in Dataset/DataFrame

2016-06-06 Thread koert kuipers (JIRA)
koert kuipers created SPARK-15798: - Summary: Secondary sort in Dataset/DataFrame Key: SPARK-15798 URL: https://issues.apache.org/jira/browse/SPARK-15798 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-14525) DataFrameWriter's save method should delegate to jdbc for jdbc datasource

2016-06-06 Thread Justin Pihony (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317846#comment-15317846 ] Justin Pihony commented on SPARK-14525: --- [~rxin] I have pushed my changes that now include

[jira] [Commented] (SPARK-15797) To expose groupingSets for DataFrame

2016-06-06 Thread Priyanka Garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317829#comment-15317829 ] Priyanka Garg commented on SPARK-15797: --- I am working on this. > To expose groupingSets for

[jira] [Created] (SPARK-15797) To expose groupingSets for DataFrame

2016-06-06 Thread Priyanka Garg (JIRA)
Priyanka Garg created SPARK-15797: - Summary: To expose groupingSets for DataFrame Key: SPARK-15797 URL: https://issues.apache.org/jira/browse/SPARK-15797 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-9221) Support IntervalType in Range Frame

2016-06-06 Thread Yang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317813#comment-15317813 ] Yang Wang commented on SPARK-9221: -- Any progress on this? [~hvanhovell] > Support IntervalType in Range

[jira] [Commented] (SPARK-15755) java.lang.NullPointerException when run spark 2.0 setting spark.serializer=org.apache.spark.serializer.KryoSerializer

2016-06-06 Thread marymwu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317782#comment-15317782 ] marymwu commented on SPARK-15755: - Any comments? > java.lang.NullPointerException when run spark 2.0

[jira] [Commented] (SPARK-15757) Error occurs when using Spark sql "select" statement on orc file after hive sql "insert overwrite tb1 select * from sourcTb" has been executed on this orc file

2016-06-06 Thread marymwu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317778#comment-15317778 ] marymwu commented on SPARK-15757: - The error occurs steps are as follows: hope it helps 1. use hive

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to

[jira] [Updated] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Feher updated SPARK-15796: Description: While debugging performance issues in a Spark program, I've found a simple way to

[jira] [Created] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

2016-06-06 Thread Gabor Feher (JIRA)
Gabor Feher created SPARK-15796: --- Summary: Spark 1.6 default memory settings can cause heavy GC when caching Key: SPARK-15796 URL: https://issues.apache.org/jira/browse/SPARK-15796 Project: Spark

[jira] [Updated] (SPARK-14485) Task finished cause fetch failure when its executor has already been removed by driver

2016-06-06 Thread iward (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] iward updated SPARK-14485: -- Summary: Task finished cause fetch failure when its executor has already been removed by driver (was: Task

[jira] [Assigned] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15795: Assignee: Apache Spark > Enable more optimizations in whole stage codegen when isNull is

[jira] [Assigned] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15795: Assignee: (was: Apache Spark) > Enable more optimizations in whole stage codegen when

[jira] [Commented] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317707#comment-15317707 ] Apache Spark commented on SPARK-15795: -- User 'inouehrs' has created a pull request for this issue:

[jira] [Created] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

2016-06-06 Thread Hiroshi Inoue (JIRA)
Hiroshi Inoue created SPARK-15795: - Summary: Enable more optimizations in whole stage codegen when isNull is a compile-time constant Key: SPARK-15795 URL: https://issues.apache.org/jira/browse/SPARK-15795

[jira] [Assigned] (SPARK-15794) Should truncate toString() of very wide schemas

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15794: Assignee: (was: Apache Spark) > Should truncate toString() of very wide schemas >

[jira] [Assigned] (SPARK-15794) Should truncate toString() of very wide schemas

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15794: Assignee: Apache Spark > Should truncate toString() of very wide schemas >

[jira] [Commented] (SPARK-15794) Should truncate toString() of very wide schemas

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317651#comment-15317651 ] Apache Spark commented on SPARK-15794: -- User 'ericl' has created a pull request for this issue:

[jira] [Created] (SPARK-15794) Should truncate toString() of very wide schemas

2016-06-06 Thread Eric Liang (JIRA)
Eric Liang created SPARK-15794: -- Summary: Should truncate toString() of very wide schemas Key: SPARK-15794 URL: https://issues.apache.org/jira/browse/SPARK-15794 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-9876) Upgrade parquet-mr to 1.8.1

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9876: --- Assignee: Ryan Blue (was: Apache Spark) > Upgrade parquet-mr to 1.8.1 >

[jira] [Assigned] (SPARK-9876) Upgrade parquet-mr to 1.8.1

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9876: --- Assignee: Apache Spark (was: Ryan Blue) > Upgrade parquet-mr to 1.8.1 >

[jira] [Commented] (SPARK-12655) GraphX does not unpersist RDDs

2016-06-06 Thread Alexander Pivovarov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317577#comment-15317577 ] Alexander Pivovarov commented on SPARK-12655: - [~srowen] Can the fix be added to 1.6 branch

[jira] [Commented] (SPARK-15574) Python meta-algorithms in Scala

2016-06-06 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317503#comment-15317503 ] Xusen Yin commented on SPARK-15574: --- [~josephkb] Can I work on this one? > Python meta-algorithms in

[jira] [Assigned] (SPARK-15793) Word2vec in ML package should have maxSentenceLength method

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15793: Assignee: Apache Spark > Word2vec in ML package should have maxSentenceLength method >

[jira] [Commented] (SPARK-15793) Word2vec in ML package should have maxSentenceLength method

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317487#comment-15317487 ] Apache Spark commented on SPARK-15793: -- User 'yinxusen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15793) Word2vec in ML package should have maxSentenceLength method

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15793: Assignee: (was: Apache Spark) > Word2vec in ML package should have maxSentenceLength

[jira] [Commented] (SPARK-14381) Review spark.ml parity for feature transformers

2016-06-06 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317459#comment-15317459 ] Xusen Yin commented on SPARK-14381: --- Comparing mllib.feature with ml.feature, there are only two APIs

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317460#comment-15317460 ] Cheng Lian commented on SPARK-15632: The {{.map(identity)}} example is quite interesting, thanks for

[jira] [Created] (SPARK-15793) Word2vec in ML package should have maxSentenceLength method

2016-06-06 Thread Xusen Yin (JIRA)
Xusen Yin created SPARK-15793: - Summary: Word2vec in ML package should have maxSentenceLength method Key: SPARK-15793 URL: https://issues.apache.org/jira/browse/SPARK-15793 Project: Spark Issue

[jira] [Resolved] (SPARK-15652) Missing org.apache.spark.launcher.SparkAppHandle.Listener notification if SparkSubmit JVM shutsdown

2016-06-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-15652. Resolution: Fixed Assignee: Subroto Sanyal Fix Version/s: 2.0.0 > Missing

[jira] [Updated] (SPARK-15791) NPE in ScalarSubquery

2016-06-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15791: --- Assignee: Eric Liang (was: Davies Liu) > NPE in ScalarSubquery > - > >

[jira] [Commented] (SPARK-14146) Imported implicits can't be found in Spark REPL in some cases

2016-06-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317383#comment-15317383 ] Shixiong Zhu commented on SPARK-14146: -- [~scrapco...@gmail.com] could you submit a PR to add it? >

[jira] [Assigned] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15792: Assignee: Apache Spark > [SQL] Allows operator to change the verbosity in explain output.

[jira] [Commented] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317379#comment-15317379 ] Apache Spark commented on SPARK-15792: -- User 'clockfly' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15792: Assignee: (was: Apache Spark) > [SQL] Allows operator to change the verbosity in

[jira] [Created] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

2016-06-06 Thread Sean Zhong (JIRA)
Sean Zhong created SPARK-15792: -- Summary: [SQL] Allows operator to change the verbosity in explain output. Key: SPARK-15792 URL: https://issues.apache.org/jira/browse/SPARK-15792 Project: Spark

[jira] [Created] (SPARK-15791) NPE in ScalarSubquery

2016-06-06 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15791: -- Summary: NPE in ScalarSubquery Key: SPARK-15791 URL: https://issues.apache.org/jira/browse/SPARK-15791 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-15716) Memory usage of driver keeps growing up in Spark Streaming

2016-06-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317340#comment-15317340 ] Shixiong Zhu commented on SPARK-15716: -- [~yani.chen] Are you using the Apache Spark or other

[jira] [Assigned] (SPARK-15789) Allow reserved keywords in most places

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15789: Assignee: (was: Apache Spark) > Allow reserved keywords in most places >

[jira] [Commented] (SPARK-15789) Allow reserved keywords in most places

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317320#comment-15317320 ] Apache Spark commented on SPARK-15789: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15789) Allow reserved keywords in most places

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15789: Assignee: Apache Spark > Allow reserved keywords in most places >

[jira] [Assigned] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15781: Assignee: Apache Spark > Misleading deprecated property in standalone cluster

[jira] [Assigned] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15781: Assignee: (was: Apache Spark) > Misleading deprecated property in standalone cluster

[jira] [Commented] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317241#comment-15317241 ] Apache Spark commented on SPARK-15781: -- User 'bomeng' has created a pull request for this issue:

[jira] [Updated] (SPARK-15782) --packages doesn't work with the spark-shell

2016-06-06 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nezih Yigitbasi updated SPARK-15782: Summary: --packages doesn't work with the spark-shell (was: --packages doesn't work the

[jira] [Created] (SPARK-15790) Audit @Since annotations in ML

2016-06-06 Thread Nick Pentreath (JIRA)
Nick Pentreath created SPARK-15790: -- Summary: Audit @Since annotations in ML Key: SPARK-15790 URL: https://issues.apache.org/jira/browse/SPARK-15790 Project: Spark Issue Type: Documentation

[jira] [Created] (SPARK-15788) PySpark IDFModel missing "idf" property

2016-06-06 Thread Nick Pentreath (JIRA)
Nick Pentreath created SPARK-15788: -- Summary: PySpark IDFModel missing "idf" property Key: SPARK-15788 URL: https://issues.apache.org/jira/browse/SPARK-15788 Project: Spark Issue Type:

[jira] [Created] (SPARK-15789) Allow reserved keywords in most places

2016-06-06 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-15789: - Summary: Allow reserved keywords in most places Key: SPARK-15789 URL: https://issues.apache.org/jira/browse/SPARK-15789 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15778. --- Resolution: Fixed > Add 2.0.0-preview to dropdown / reorg description of previews at >

[jira] [Assigned] (SPARK-15204) Improve nullability inference for Aggregator

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15204: Assignee: (was: Apache Spark) > Improve nullability inference for Aggregator >

[jira] [Commented] (SPARK-15204) Improve nullability inference for Aggregator

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317187#comment-15317187 ] Apache Spark commented on SPARK-15204: -- User 'koertkuipers' has created a pull request for this

[jira] [Assigned] (SPARK-15204) Improve nullability inference for Aggregator

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15204: Assignee: Apache Spark > Improve nullability inference for Aggregator >

[jira] [Commented] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317178#comment-15317178 ] Apache Spark commented on SPARK-15654: -- User 'davies' has created a pull request for this issue:

[jira] [Commented] (SPARK-7960) Serialization problem when multiple receivers are specified in a loop

2016-06-06 Thread UTKARSH BHATNAGAR (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317176#comment-15317176 ] UTKARSH BHATNAGAR commented on SPARK-7960: -- DEBUG Output: 16/06/06 20:29:00 DEBUG

[jira] [Reopened] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-15778: --- One more comment from [~rxin] -- maybe put the preview releases in a separate section in the dropdown.

[jira] [Updated] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15778: -- Attachment: Screen Shot 2016-06-06 at 21.25.59.png Now with Stable/Preview section. It's sort of funny

[jira] [Commented] (SPARK-15507) ClassCastException: SomeCaseClass cannot be cast to org.apache.spark.sql.Row

2016-06-06 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317156#comment-15317156 ] koert kuipers commented on SPARK-15507: --- also soon we will not try to do this in a RDD anymore

[jira] [Commented] (SPARK-15783) Fix more flakiness: o.a.s.scheduler.BlacklistIntegrationSuite

2016-06-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317155#comment-15317155 ] Imran Rashid commented on SPARK-15783: -- Turned the tests off via

[jira] [Commented] (SPARK-15069) GSoC 2016: Exposing more R and Python APIs for MLlib

2016-06-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317151#comment-15317151 ] Joseph K. Bradley commented on SPARK-15069: --- h4. 6/6/2016 - Week 3 To-do items * Continuation

[jira] [Updated] (SPARK-15767) Decision Tree Regression wrapper in SparkR

2016-06-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15767: -- Assignee: Kai Jiang Affects Version/s: (was: 2.1.0) Target

[jira] [Commented] (SPARK-15672) R programming guide update

2016-06-06 Thread Kai Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317133#comment-15317133 ] Kai Jiang commented on SPARK-15672: --- Working on it. > R programming guide update >

[jira] [Commented] (SPARK-15507) ClassCastException: SomeCaseClass cannot be cast to org.apache.spark.sql.Row

2016-06-06 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317131#comment-15317131 ] koert kuipers commented on SPARK-15507: --- we do not know all the columns, but we do not want to drop

[jira] [Commented] (SPARK-7960) Serialization problem when multiple receivers are specified in a loop

2016-06-06 Thread UTKARSH BHATNAGAR (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317103#comment-15317103 ] UTKARSH BHATNAGAR commented on SPARK-7960: -- This problem persists in Spark 1.6.1 too , have

[jira] [Resolved] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15778. --- Resolution: Fixed Fix Version/s: 2.0.0 Live at http://spark.apache.org/downloads.html > Add

[jira] [Commented] (SPARK-15507) ClassCastException: SomeCaseClass cannot be cast to org.apache.spark.sql.Row

2016-06-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317084#comment-15317084 ] Wenchen Fan commented on SPARK-15507: - if you have a RDD[Row], just call `rdd.toDF().as[YourClass]`,

[jira] [Assigned] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15654: -- Assignee: Davies Liu (was: Takeshi Yamamuro) > Reading gzipped files results in duplicate

[jira] [Commented] (SPARK-14279) Improve the spark build to pick the version information from the pom file and add git commit information

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317005#comment-15317005 ] Apache Spark commented on SPARK-14279: -- User 'dhruve' has created a pull request for this issue:

[jira] [Updated] (SPARK-15786) joinWith bytecode generation calling ByteBuffer.wrap with InternalRow

2016-06-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-15786: - Target Version/s: 2.0.0 > joinWith bytecode generation calling ByteBuffer.wrap with

[jira] [Created] (SPARK-15787) Display more helpful error messages for several invalid operations

2016-06-06 Thread nalin garg (JIRA)
nalin garg created SPARK-15787: -- Summary: Display more helpful error messages for several invalid operations Key: SPARK-15787 URL: https://issues.apache.org/jira/browse/SPARK-15787 Project: Spark

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316998#comment-15316998 ] Sean Zhong commented on SPARK-15632: *Root cause analysis:* *The root cause is that the

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316996#comment-15316996 ] Sean Zhong commented on SPARK-15632: There are more issues linked with this bug which we may to fix

[jira] [Created] (SPARK-15786) Spark SQL - joinWith bytecode generation calling ByteBuffer.wrap with InternalRow

2016-06-06 Thread Richard Marscher (JIRA)
Richard Marscher created SPARK-15786: Summary: Spark SQL - joinWith bytecode generation calling ByteBuffer.wrap with InternalRow Key: SPARK-15786 URL: https://issues.apache.org/jira/browse/SPARK-15786

[jira] [Updated] (SPARK-15786) joinWith bytecode generation calling ByteBuffer.wrap with InternalRow

2016-06-06 Thread Richard Marscher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Marscher updated SPARK-15786: - Summary: joinWith bytecode generation calling ByteBuffer.wrap with InternalRow (was:

[jira] [Assigned] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15632: Assignee: Xiang Zhong (was: Apache Spark) > Dataset typed filter operation changes query

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316979#comment-15316979 ] Apache Spark commented on SPARK-15632: -- User 'clockfly' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15632: Assignee: Apache Spark (was: Xiang Zhong) > Dataset typed filter operation changes query

[jira] [Resolved] (SPARK-15764) Replace n^2 loop in BindReferences

2016-06-06 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-15764. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13505

[jira] [Updated] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15654: --- Assignee: Takeshi Yamamuro > Reading gzipped files results in duplicate rows >

[jira] [Commented] (SPARK-15780) Support mapValues on KeyValueGroupedDataset

2016-06-06 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316943#comment-15316943 ] koert kuipers commented on SPARK-15780: --- original discussion is here:

[jira] [Commented] (SPARK-11159) Nested SQL UDF raises java.lang.UnsupportedOperationException: Cannot evaluate expression

2016-06-06 Thread Christoph Tavan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316922#comment-15316922 ] Christoph Tavan commented on SPARK-11159: - I can confirm this bug with spark 1.6.1 on databricks.

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

2016-06-06 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316908#comment-15316908 ] Shivaram Venkataraman commented on SPARK-13525: --- [~vmenda] Your stack trace indicates that

[jira] [Comment Edited] (SPARK-14767) Codegen "no constructor found" errors with Maps inside case classes in Datasets

2016-06-06 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316904#comment-15316904 ] Burak Yavuz edited comment on SPARK-14767 at 6/6/16 6:11 PM: - I still run

[jira] [Commented] (SPARK-14767) Codegen "no constructor found" errors with Maps inside case classes in Datasets

2016-06-06 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316904#comment-15316904 ] Burak Yavuz commented on SPARK-14767: - I still run into this > Codegen "no constructor found" errors

[jira] [Commented] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316899#comment-15316899 ] Reynold Xin commented on SPARK-15778: - See https://issues.apache.org/jira/browse/INFRA-12051 for the

[jira] [Commented] (SPARK-15159) SparkSession R API

2016-06-06 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316898#comment-15316898 ] Shivaram Venkataraman commented on SPARK-15159: --- That sounds good. Also one way to handle

[jira] [Updated] (SPARK-8426) Add blacklist mechanism for YARN container allocation

2016-06-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-8426: Attachment: DesignDocforBlacklistMechanism.pdf Updated Design Doc, based on performance tests and

[jira] [Comment Edited] (SPARK-15687) Columnar execution engine

2016-06-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316858#comment-15316858 ] Kazuaki Ishizaki edited comment on SPARK-15687 at 6/6/16 5:49 PM: -- Thank

[jira] [Commented] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316865#comment-15316865 ] Reynold Xin commented on SPARK-15778: - LGTM. Maybe we should request INFRA to convert the website

[jira] [Comment Edited] (SPARK-15778) Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html

2016-06-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316865#comment-15316865 ] Reynold Xin edited comment on SPARK-15778 at 6/6/16 5:49 PM: - LGTM. Maybe we

[jira] [Commented] (SPARK-15687) Columnar execution engine

2016-06-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316858#comment-15316858 ] Kazuaki Ishizaki commented on SPARK-15687: -- Thank you for your answers * How we pass columnar

[jira] [Commented] (SPARK-13174) Add API and options for csv data sources

2016-06-06 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316848#comment-15316848 ] Shivaram Venkataraman commented on SPARK-13174: --- +1 for `read.df("path/to/file.csv", source

[jira] [Commented] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

2016-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316838#comment-15316838 ] Sean Owen commented on SPARK-15781: --- It's probably OK to document them as long as they actually have an

[jira] [Comment Edited] (SPARK-14380) Review spark.ml parity for clustering

2016-06-06 Thread Xinh Huynh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314617#comment-15314617 ] Xinh Huynh edited comment on SPARK-14380 at 6/6/16 5:29 PM: Existing

[jira] [Created] (SPARK-15785) Add initialModel param to Gaussian Mixture Model (GMM) in spark.ml

2016-06-06 Thread Xinh Huynh (JIRA)
Xinh Huynh created SPARK-15785: -- Summary: Add initialModel param to Gaussian Mixture Model (GMM) in spark.ml Key: SPARK-15785 URL: https://issues.apache.org/jira/browse/SPARK-15785 Project: Spark

[jira] [Comment Edited] (SPARK-14380) Review spark.ml parity for clustering

2016-06-06 Thread Xinh Huynh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314345#comment-15314345 ] Xinh Huynh edited comment on SPARK-14380 at 6/6/16 5:26 PM: Completely

  1   2   >