[jira] [Assigned] (SPARK-14856) Returning batch unexpected from wide table

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14856: Assignee: Apache Spark (was: Davies Liu) > Returning batch unexpected from wide table >

[jira] [Commented] (SPARK-14857) Table/Database Name Validation in SessionCatalog

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254575#comment-15254575 ] Apache Spark commented on SPARK-14857: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14857) Table/Database Name Validation in SessionCatalog

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14857: Assignee: Apache Spark > Table/Database Name Validation in SessionCatalog >

[jira] [Assigned] (SPARK-14857) Table/Database Name Validation in SessionCatalog

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14857: Assignee: (was: Apache Spark) > Table/Database Name Validation in SessionCatalog >

[jira] [Resolved] (SPARK-10129) math function: stddev_samp

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10129. Resolution: Not A Problem > math function: stddev_samp > -- > >

[jira] [Created] (SPARK-14857) Table/Database Name Validation in SessionCatalog

2016-04-22 Thread Xiao Li (JIRA)
Xiao Li created SPARK-14857: --- Summary: Table/Database Name Validation in SessionCatalog Key: SPARK-14857 URL: https://issues.apache.org/jira/browse/SPARK-14857 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-10600) SparkSQL - Support for Not Exists in a Correlated Subquery

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10600. Resolution: Duplicate Assignee: Herman van Hovell Fix Version/s: 2.0.0 > SparkSQL

[jira] [Resolved] (SPARK-13831) TPC-DS Query 35 fails with the following compile error

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13831. Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 2.0.0 > TPC-DS Query

[jira] [Resolved] (SPARK-12545) Support exists condition

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12545. Resolution: Duplicate Assignee: Herman van Hovell (was: Davies Liu) Fix Version/s:

[jira] [Resolved] (SPARK-13347) Reuse the shuffle for duplicated exchange

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13347. Resolution: Duplicate Assignee: Davies Liu Fix Version/s: 2.0.0 > Reuse the

[jira] [Resolved] (SPARK-13348) Avoid duplicated broadcasts

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13348. Resolution: Fixed Assignee: Davies Liu Fix Version/s: 2.0.0 It's fixed by re-use

[jira] [Closed] (SPARK-13541) Flaky test: ParquetHadoopFsRelationSuite.test all data types - ByteType

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-13541. -- Resolution: Cannot Reproduce > Flaky test: ParquetHadoopFsRelationSuite.test all data types - ByteType

[jira] [Resolved] (SPARK-14669) Some SQL metrics is broken when whole-stage codegen enabled

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14669. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12425

[jira] [Commented] (SPARK-14831) Make ML APIs in SparkR consistent

2016-04-22 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254551#comment-15254551 ] Shivaram Venkataraman commented on SPARK-14831: --- 1. Agree. I think a valid policy could be

[jira] [Updated] (SPARK-14850) VectorUDT/MatrixUDT should take primitive arrays without boxing

2016-04-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14850: -- Priority: Blocker (was: Critical) > VectorUDT/MatrixUDT should take primitive arrays without

[jira] [Commented] (SPARK-9478) Add class weights to Random Forest

2016-04-22 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254539#comment-15254539 ] Seth Hendrickson commented on SPARK-9478: - [~josephkb] I have a PR ready for this. It's being

[jira] [Updated] (SPARK-14850) VectorUDT/MatrixUDT should take primitive arrays without boxing

2016-04-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14850: -- Affects Version/s: 1.5.2 > VectorUDT/MatrixUDT should take primitive arrays without boxing >

[jira] [Commented] (SPARK-14850) VectorUDT/MatrixUDT should take primitive arrays without boxing

2016-04-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254538#comment-15254538 ] Xiangrui Meng commented on SPARK-14850: --- Ran the following code with different Spark versions:

[jira] [Assigned] (SPARK-14855) Add "Exec" suffix to all physical operators

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14855: Assignee: Apache Spark (was: Reynold Xin) > Add "Exec" suffix to all physical operators

[jira] [Commented] (SPARK-14855) Add "Exec" suffix to all physical operators

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254529#comment-15254529 ] Apache Spark commented on SPARK-14855: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-14856) Returning batch unexpected from wide table

2016-04-22 Thread Davies Liu (JIRA)
Davies Liu created SPARK-14856: -- Summary: Returning batch unexpected from wide table Key: SPARK-14856 URL: https://issues.apache.org/jira/browse/SPARK-14856 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-14855) Add "Exec" suffix to all physical operators

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14855: Assignee: Reynold Xin (was: Apache Spark) > Add "Exec" suffix to all physical operators

[jira] [Commented] (SPARK-14831) Make ML APIs in SparkR consistent

2016-04-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254516#comment-15254516 ] Xiangrui Meng commented on SPARK-14831: --- 1. Please see my reply to Felix above for the issue with

[jira] [Created] (SPARK-14855) Add "Exec" suffix to all physical operators

2016-04-22 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-14855: --- Summary: Add "Exec" suffix to all physical operators Key: SPARK-14855 URL: https://issues.apache.org/jira/browse/SPARK-14855 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-14791) TPCDS Q23B generate different result each time

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14791. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12600

[jira] [Commented] (SPARK-14831) Make ML APIs in SparkR consistent

2016-04-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254510#comment-15254510 ] Xiangrui Meng commented on SPARK-14831: --- We have been trying to mimic existing R APIs in SparkR.

[jira] [Updated] (SPARK-14854) Left outer join produces incorrect output when the join condition does not have left table key

2016-04-22 Thread kanika dhuria (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kanika dhuria updated SPARK-14854: -- Description: import org.apache.spark.sql._ import org.apache.spark.sql.types._ val s =

[jira] [Created] (SPARK-14854) Left outer join produces incorrect output when the join condition does not have left table key

2016-04-22 Thread kanika dhuria (JIRA)
kanika dhuria created SPARK-14854: - Summary: Left outer join produces incorrect output when the join condition does not have left table key Key: SPARK-14854 URL: https://issues.apache.org/jira/browse/SPARK-14854

[jira] [Resolved] (SPARK-14763) Can't analyze TPCDS Q70

2016-04-22 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-14763. --- Resolution: Fixed Fix Version/s: 2.0.0 > Can't analyze TPCDS Q70 >

[jira] [Created] (SPARK-14853) Support LeftSemi/LeftAnti in SortMergeJoin

2016-04-22 Thread Davies Liu (JIRA)
Davies Liu created SPARK-14853: -- Summary: Support LeftSemi/LeftAnti in SortMergeJoin Key: SPARK-14853 URL: https://issues.apache.org/jira/browse/SPARK-14853 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-14837) Add support in file stream source for reading new files added to subdirs

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14837: Assignee: Tathagata Das (was: Apache Spark) > Add support in file stream source for

[jira] [Commented] (SPARK-14831) Make ML APIs in SparkR consistent

2016-04-22 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254447#comment-15254447 ] Shivaram Venkataraman commented on SPARK-14831: --- Yeah I think there are a couple of factors

[jira] [Commented] (SPARK-14837) Add support in file stream source for reading new files added to subdirs

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254450#comment-15254450 ] Apache Spark commented on SPARK-14837: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14837) Add support in file stream source for reading new files added to subdirs

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14837: Assignee: Apache Spark (was: Tathagata Das) > Add support in file stream source for

[jira] [Updated] (SPARK-14843) Error while encoding: java.lang.ClassCastException with LibSVMRelation

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14843: -- Assignee: Liang-Chi Hsieh > Error while encoding: java.lang.ClassCastException with LibSVMRelation >

[jira] [Resolved] (SPARK-14762) Fail to parse TPCDS Q90

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14762. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12537

[jira] [Commented] (SPARK-14842) Implement view creation in sql/core

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254410#comment-15254410 ] Apache Spark commented on SPARK-14842: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-14852) Update GeneralizedLinearRegressionSummary API

2016-04-22 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-14852: - Summary: Update GeneralizedLinearRegressionSummary API Key: SPARK-14852 URL: https://issues.apache.org/jira/browse/SPARK-14852 Project: Spark

[jira] [Updated] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-04-22 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-13178: -- Assignee: Sun Rui > RRDD faces with concurrency issue in case of

[jira] [Commented] (SPARK-14817) ML 2.0 QA: Programming guide update and migration guide

2016-04-22 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254371#comment-15254371 ] Xin Ren commented on SPARK-14817: - cout me too :) > ML 2.0 QA: Programming guide update and migration

[jira] [Resolved] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-04-22 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-13178. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request

[jira] [Resolved] (SPARK-14841) Move SQLBuilder into sql/core

2016-04-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14841. - Resolution: Fixed Fix Version/s: 2.0.0 > Move SQLBuilder into sql/core >

[jira] [Comment Edited] (SPARK-14694) Thrift Server + Hive Metastore + Kerberos doesn't work

2016-04-22 Thread Andrew Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254337#comment-15254337 ] Andrew Lee edited comment on SPARK-14694 at 4/22/16 6:03 PM: - Not sure if

[jira] [Commented] (SPARK-14694) Thrift Server + Hive Metastore + Kerberos doesn't work

2016-04-22 Thread Andrew Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254337#comment-15254337 ] Andrew Lee commented on SPARK-14694: I'm able to get this working with Spark 1.6.1 + Hive 1.2 +

[jira] [Created] (SPARK-14851) Support radix sort with nullable longs

2016-04-22 Thread Eric Liang (JIRA)
Eric Liang created SPARK-14851: -- Summary: Support radix sort with nullable longs Key: SPARK-14851 URL: https://issues.apache.org/jira/browse/SPARK-14851 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-14818) Move sketch, mllibLocal, and hivecontext-compatibility out from mima exclusion

2016-04-22 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-14818: - Summary: Move sketch, mllibLocal, and hivecontext-compatibility out from mima exclusion (was: Move

[jira] [Resolved] (SPARK-14843) Error while encoding: java.lang.ClassCastException with LibSVMRelation

2016-04-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14843. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12611

[jira] [Updated] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14848: -- Assignee: Pete Robbins > DatasetSuite - Java encoder fails on Big Endian platforms >

[jira] [Updated] (SPARK-13928) Move org.apache.spark.Logging into org.apache.spark.internal.Logging

2016-04-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13928: --- Target Version/s: 2.0.0 > Move org.apache.spark.Logging into org.apache.spark.internal.Logging >

[jira] [Updated] (SPARK-13928) Move org.apache.spark.Logging into org.apache.spark.internal.Logging

2016-04-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13928: --- Assignee: Wenchen Fan > Move org.apache.spark.Logging into org.apache.spark.internal.Logging >

[jira] [Updated] (SPARK-13928) Move org.apache.spark.Logging into org.apache.spark.internal.Logging

2016-04-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13928: --- Fix Version/s: 2.0.0 > Move org.apache.spark.Logging into org.apache.spark.internal.Logging >

[jira] [Assigned] (SPARK-14604) Modify design of ML model summaries

2016-04-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-14604: - Assignee: Joseph K. Bradley > Modify design of ML model summaries >

[jira] [Commented] (SPARK-14817) ML 2.0 QA: Programming guide update and migration guide

2016-04-22 Thread Benjamin Fradet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254196#comment-15254196 ] Benjamin Fradet commented on SPARK-14817: - Count me in! > ML 2.0 QA: Programming guide update

[jira] [Updated] (SPARK-7768) Make user-defined type (UDT) API public

2016-04-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7768: --- Target Version/s: 2.1.0 (was: 2.0.0) > Make user-defined type (UDT) API public >

[jira] [Closed] (SPARK-922) Update Spark AMI to Python 2.7

2016-04-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-922. - Resolution: Not A Problem This is now outside the scope of Spark. > Update Spark AMI to Python 2.7 >

[jira] [Created] (SPARK-14850) VectorUDT/MatrixUDT should take primitive arrays without boxing

2016-04-22 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-14850: - Summary: VectorUDT/MatrixUDT should take primitive arrays without boxing Key: SPARK-14850 URL: https://issues.apache.org/jira/browse/SPARK-14850 Project: Spark

[jira] [Updated] (SPARK-14541) SQL function: IFNULL, NULLIF, NVL and NVL2

2016-04-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14541: Target Version/s: 2.0.0 > SQL function: IFNULL, NULLIF, NVL and NVL2 >

[jira] [Updated] (SPARK-14850) VectorUDT/MatrixUDT should take primitive arrays without boxing

2016-04-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14850: -- Description: In SPARK-9390, we switched to use GenericArrayData to store indices and values

[jira] [Assigned] (SPARK-14730) Expose ColumnPruner as feature transformer

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14730: Assignee: (was: Apache Spark) > Expose ColumnPruner as feature transformer >

[jira] [Assigned] (SPARK-14730) Expose ColumnPruner as feature transformer

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14730: Assignee: Apache Spark > Expose ColumnPruner as feature transformer >

[jira] [Commented] (SPARK-14730) Expose ColumnPruner as feature transformer

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254181#comment-15254181 ] Apache Spark commented on SPARK-14730: -- User 'BenFradet' has created a pull request for this issue:

[jira] [Updated] (SPARK-13266) Python DataFrameReader converts None to "None" instead of null

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13266: --- Assignee: Liang-Chi Hsieh > Python DataFrameReader converts None to "None" instead of null >

[jira] [Resolved] (SPARK-13266) Python DataFrameReader converts None to "None" instead of null

2016-04-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13266. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12494

[jira] [Assigned] (SPARK-14849) shuffle broken when accessing standalone cluster through NAT

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14849: Assignee: Apache Spark > shuffle broken when accessing standalone cluster through NAT >

[jira] [Assigned] (SPARK-14849) shuffle broken when accessing standalone cluster through NAT

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14849: Assignee: (was: Apache Spark) > shuffle broken when accessing standalone cluster

[jira] [Commented] (SPARK-14849) shuffle broken when accessing standalone cluster through NAT

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254082#comment-15254082 ] Apache Spark commented on SPARK-14849: -- User 'skyluc' has created a pull request for this issue:

[jira] [Resolved] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms

2016-04-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-14848. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12610

[jira] [Comment Edited] (SPARK-14849) shuffle broken when accessing standalone cluster through NAT

2016-04-22 Thread Luc Bourlier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253968#comment-15253968 ] Luc Bourlier edited comment on SPARK-14849 at 4/22/16 3:07 PM: --- I have dug

[jira] [Commented] (SPARK-14138) Generated SpecificColumnarIterator code can exceed JVM size limit for cached DataFrames

2016-04-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254061#comment-15254061 ] Kazuaki Ishizaki commented on SPARK-14138: -- Yes, it will be included in 1.6.2 and 2.0.0. >

[jira] [Commented] (SPARK-14654) New accumulator API

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254053#comment-15254053 ] Apache Spark commented on SPARK-14654: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14654) New accumulator API

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14654: Assignee: Apache Spark > New accumulator API > --- > >

[jira] [Assigned] (SPARK-14654) New accumulator API

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14654: Assignee: (was: Apache Spark) > New accumulator API > --- > >

[jira] [Comment Edited] (SPARK-14831) Make ML APIs in SparkR consistent

2016-04-22 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254032#comment-15254032 ] Yanbo Liang edited comment on SPARK-14831 at 4/22/16 2:37 PM: -- This change

[jira] [Commented] (SPARK-14831) Make ML APIs in SparkR consistent

2016-04-22 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254032#comment-15254032 ] Yanbo Liang commented on SPARK-14831: - This change looks good to me. Thanks! BTW, I think we should

[jira] [Commented] (SPARK-14138) Generated SpecificColumnarIterator code can exceed JVM size limit for cached DataFrames

2016-04-22 Thread William Kinney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253976#comment-15253976 ] William Kinney commented on SPARK-14138: Is there a workaround for this for 1.6.1? > Generated

[jira] [Commented] (SPARK-14849) shuffle broken when accessing standalone cluster through NAT

2016-04-22 Thread Luc Bourlier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253968#comment-15253968 ] Luc Bourlier commented on SPARK-14849: -- I have dug at the problem. It is created during the

[jira] [Assigned] (SPARK-14843) Error while encoding: java.lang.ClassCastException with LibSVMRelation

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14843: Assignee: (was: Apache Spark) > Error while encoding: java.lang.ClassCastException

[jira] [Commented] (SPARK-14843) Error while encoding: java.lang.ClassCastException with LibSVMRelation

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253960#comment-15253960 ] Apache Spark commented on SPARK-14843: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14843) Error while encoding: java.lang.ClassCastException with LibSVMRelation

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14843: Assignee: Apache Spark > Error while encoding: java.lang.ClassCastException with

[jira] [Created] (SPARK-14849) shuffle broken when accessing standalone cluster through NAT

2016-04-22 Thread Luc Bourlier (JIRA)
Luc Bourlier created SPARK-14849: Summary: shuffle broken when accessing standalone cluster through NAT Key: SPARK-14849 URL: https://issues.apache.org/jira/browse/SPARK-14849 Project: Spark

[jira] [Assigned] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14848: Assignee: (was: Apache Spark) > DatasetSuite - Java encoder fails on Big Endian

[jira] [Commented] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253826#comment-15253826 ] Apache Spark commented on SPARK-14848: -- User 'robbinspg' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14848: Assignee: Apache Spark > DatasetSuite - Java encoder fails on Big Endian platforms >

[jira] [Commented] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms

2016-04-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253821#comment-15253821 ] Wenchen Fan commented on SPARK-14848: - Yea, according to SQL specific, the result order of

[jira] [Updated] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms

2016-04-22 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pete Robbins updated SPARK-14848: - Description: Since this PR https://github.com/apache/spark/pull/10703 for

[jira] [Commented] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms

2016-04-22 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253816#comment-15253816 ] Pete Robbins commented on SPARK-14848: -- changing the Java encoder test to use toSet and compare

[jira] [Updated] (SPARK-6717) Clear shuffle files after checkpointing in ALS

2016-04-22 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-6717: -- Shepherd: Nick Pentreath > Clear shuffle files after checkpointing in ALS >

[jira] [Updated] (SPARK-14754) Metrics as logs are not coming through slf4j

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14754: -- Flags: (was: Patch) Target Version/s: (was: 1.6.2) Labels: (was:

[jira] [Updated] (SPARK-14579) Fix a race condition in StreamExecution.processAllAvailable

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14579: -- Fix Version/s: (was: 2.0.0) > Fix a race condition in StreamExecution.processAllAvailable >

[jira] [Updated] (SPARK-14779) Incorrect log message in Worker while handling KillExecutor message

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14779: -- Assignee: Bryan Cutler > Incorrect log message in Worker while handling KillExecutor message >

[jira] [Updated] (SPARK-13842) Consider __iter__ and __getitem__ methods for pyspark.sql.types.StructType

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13842: -- Assignee: Shea Parkes > Consider __iter__ and __getitem__ methods for pyspark.sql.types.StructType >

[jira] [Created] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms

2016-04-22 Thread Pete Robbins (JIRA)
Pete Robbins created SPARK-14848: Summary: DatasetSuite - Java encoder fails on Big Endian platforms Key: SPARK-14848 URL: https://issues.apache.org/jira/browse/SPARK-14848 Project: Spark

[jira] [Updated] (SPARK-14799) Remove MetastoreRelation dependency from AnalyzeTable

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14799: -- Assignee: Reynold Xin > Remove MetastoreRelation dependency from AnalyzeTable >

[jira] [Updated] (SPARK-14724) Improve performance of sorting by using radix sort when possible

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14724: -- Assignee: Eric Liang > Improve performance of sorting by using radix sort when possible >

[jira] [Commented] (SPARK-14847) ML/MLlib breaking changes between 1.6 & 2.0

2016-04-22 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253774#comment-15253774 ] Yanbo Liang commented on SPARK-14847: - [~sowen] Sorry, I did not found out SPARK-13448. I will close

[jira] [Updated] (SPARK-6429) Add to style checker "hashCode and equals should be defined together"

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6429: - Assignee: Joan Goyeau > Add to style checker "hashCode and equals should be defined together" >

[jira] [Updated] (SPARK-14847) ML/MLlib breaking changes between 1.6 & 2.0

2016-04-22 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-14847: Description: This PR records the breaking changes of ML/MLlib between 1.6 and 2.0, so we can note

[jira] [Resolved] (SPARK-6429) Add to style checker "hashCode and equals should be defined together"

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6429. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12157

[jira] [Resolved] (SPARK-14847) ML/MLlib breaking changes between 1.6 & 2.0

2016-04-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14847. --- Resolution: Duplicate Arguably duplicates SPARK-12626, SPARK-14808, and non-MLlib specific 'roadmap'

[jira] [Assigned] (SPARK-14844) KMeansModel in spark.ml should allow to change featureCol and predictionCol

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14844: Assignee: (was: Apache Spark) > KMeansModel in spark.ml should allow to change

<    1   2   3   >