[jira] [Resolved] (SPARK-16888) Implements eval method for expression AssertNotNull

2016-08-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16888. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14486

[jira] [Updated] (SPARK-16888) Implements eval method for expression AssertNotNull

2016-08-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16888: Assignee: Sean Zhong > Implements eval method for expression AssertNotNull >

[jira] [Commented] (SPARK-16876) Add match Column expression for regular expression matching in Scala API

2016-08-03 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407200#comment-15407200 ] Kapil Singh commented on SPARK-16876: - Yes these are similar, but SPARK-16203 is for PySpark while

[jira] [Commented] (SPARK-16798) java.lang.IllegalArgumentException: bound must be positive : Worked in 1.5.2

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407104#comment-15407104 ] Sean Owen commented on SPARK-16798: --- That's odd to be sure. It's as if the distributePartition function

[jira] [Commented] (SPARK-13710) Spark shell shows ERROR when launching on Windows

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407096#comment-15407096 ] Sean Owen commented on SPARK-13710: --- Possibly related to

[jira] [Commented] (SPARK-16888) Implements eval method for expression AssertNotNull

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407077#comment-15407077 ] Apache Spark commented on SPARK-16888: -- User 'clockfly' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16888) Implements eval method for expression AssertNotNull

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16888: Assignee: (was: Apache Spark) > Implements eval method for expression AssertNotNull >

[jira] [Assigned] (SPARK-16888) Implements eval method for expression AssertNotNull

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16888: Assignee: Apache Spark > Implements eval method for expression AssertNotNull >

[jira] [Commented] (SPARK-16885) Spark shell failed to run in yarn-client mode

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407073#comment-15407073 ] Sean Owen commented on SPARK-16885: --- I'm pretty certain it's because you specified a directory, not a

[jira] [Created] (SPARK-16888) Implements eval method for expression AssertNotNull

2016-08-03 Thread Sean Zhong (JIRA)
Sean Zhong created SPARK-16888: -- Summary: Implements eval method for expression AssertNotNull Key: SPARK-16888 URL: https://issues.apache.org/jira/browse/SPARK-16888 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16887) Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPATH

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407053#comment-15407053 ] Apache Spark commented on SPARK-16887: -- User 'yhuai' has created a pull request for this issue:

[jira] [Commented] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-08-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407043#comment-15407043 ] Hyukjin Kwon commented on SPARK-16610: -- Actually, my initial proposal of the above was including

[jira] [Assigned] (SPARK-16887) Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPATH

2016-08-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai reassigned SPARK-16887: Assignee: Yin Huai > Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPATH >

[jira] [Created] (SPARK-16887) Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPATH

2016-08-03 Thread Yin Huai (JIRA)
Yin Huai created SPARK-16887: Summary: Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPATH Key: SPARK-16887 URL: https://issues.apache.org/jira/browse/SPARK-16887 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16886) StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset

2016-08-03 Thread Ganesh Chand (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407031#comment-15407031 ] Ganesh Chand commented on SPARK-16886: -- Submitted a pull request -

[jira] [Assigned] (SPARK-16886) StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16886: Assignee: (was: Apache Spark) > StructuredNetworkWordCount code comment incorrectly

[jira] [Commented] (SPARK-16886) StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407032#comment-15407032 ] Apache Spark commented on SPARK-16886: -- User 'ganeshchand' has created a pull request for this

[jira] [Assigned] (SPARK-16886) StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16886: Assignee: Apache Spark > StructuredNetworkWordCount code comment incorrectly refers to

[jira] [Assigned] (SPARK-16877) Add a rule for preventing use Java's Override annotation

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16877: Assignee: (was: Apache Spark) > Add a rule for preventing use Java's Override

[jira] [Assigned] (SPARK-16877) Add a rule for preventing use Java's Override annotation

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16877: Assignee: Apache Spark > Add a rule for preventing use Java's Override annotation >

[jira] [Commented] (SPARK-16877) Add a rule for preventing use Java's Override annotation

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407025#comment-15407025 ] Apache Spark commented on SPARK-16877: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-16873) force spill NPE

2016-08-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16873. - Resolution: Fixed Assignee: sharkd tu Fix Version/s: 2.1.0

[jira] [Updated] (SPARK-16886) StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset

2016-08-03 Thread Ganesh Chand (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ganesh Chand updated SPARK-16886: - Description: Both StructuredNetworkWordCount.scala and StructuredNetworkWordCount.java has the

[jira] [Updated] (SPARK-16886) StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset

2016-08-03 Thread Ganesh Chand (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ganesh Chand updated SPARK-16886: - Description: Both StructuredNetworkWordCount.scala and StructuredNetworkWordCount.java has the

[jira] [Comment Edited] (SPARK-16798) java.lang.IllegalArgumentException: bound must be positive : Worked in 1.5.2

2016-08-03 Thread Charles Allen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406967#comment-15406967 ] Charles Allen edited comment on SPARK-16798 at 8/4/16 1:41 AM: --- [~srowen] I

[jira] [Commented] (SPARK-16798) java.lang.IllegalArgumentException: bound must be positive : Worked in 1.5.2

2016-08-03 Thread Charles Allen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406967#comment-15406967 ] Charles Allen commented on SPARK-16798: --- [~srowen] I manually went in with IntelliJ debugging and

[jira] [Created] (SPARK-16886) StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset

2016-08-03 Thread Ganesh Chand (JIRA)
Ganesh Chand created SPARK-16886: Summary: StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset Key: SPARK-16886 URL: https://issues.apache.org/jira/browse/SPARK-16886

[jira] [Updated] (SPARK-16885) Spark shell failed to run in yarn-client mode

2016-08-03 Thread Yury Zhyshko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Zhyshko updated SPARK-16885: - Attachment: spark-env.sh > Spark shell failed to run in yarn-client mode >

[jira] [Updated] (SPARK-16885) Spark shell failed to run in yarn-client mode

2016-08-03 Thread Yury Zhyshko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Zhyshko updated SPARK-16885: - Description: I've installed Hadoop + Yarn in pseudo distributed mode following these

[jira] [Created] (SPARK-16885) Spark shell failed to run in yarn-client mode

2016-08-03 Thread Yury Zhyshko (JIRA)
Yury Zhyshko created SPARK-16885: Summary: Spark shell failed to run in yarn-client mode Key: SPARK-16885 URL: https://issues.apache.org/jira/browse/SPARK-16885 Project: Spark Issue Type:

[jira] [Updated] (SPARK-16814) Fix deprecated use of ParquetWriter in Parquet test suites

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16814: -- Priority: Minor (was: Major) > Fix deprecated use of ParquetWriter in Parquet test suites >

[jira] [Assigned] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16826: Assignee: (was: Apache Spark) > java.util.Hashtable limits the throughput of

[jira] [Commented] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406868#comment-15406868 ] Apache Spark commented on SPARK-16826: -- User 'sylvinus' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16826: Assignee: Apache Spark > java.util.Hashtable limits the throughput of PARSE_URL() >

[jira] [Updated] (SPARK-16814) Fix deprecated use of ParquetWriter in Parquet test suites

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16814: -- Assignee: holdenk > Fix deprecated use of ParquetWriter in Parquet test suites >

[jira] [Resolved] (SPARK-16814) Fix deprecated use of ParquetWriter in Parquet test suites

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16814. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14419

[jira] [Updated] (SPARK-16770) Spark shell not usable with german keyboard due to JLine version

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16770: -- Assignee: Stefan Schulze > Spark shell not usable with german keyboard due to JLine version >

[jira] [Resolved] (SPARK-16770) Spark shell not usable with german keyboard due to JLine version

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16770. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Updated] (SPARK-16870) add "spark.sql.broadcastTimeout" into docs/sql-programming-guide.md to help people to how to fix this timeout error when it happenned

2016-08-03 Thread Liang Ke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Ke updated SPARK-16870: - Description: here my workload and what I found I run a large number jobs with spark-sql at the same

[jira] [Updated] (SPARK-16870) add "spark.sql.broadcastTimeout" into docs/sql-programming-guide.md to help people to how to fix this timeout error when it happenned

2016-08-03 Thread Liang Ke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Ke updated SPARK-16870: - Description: here my workload and what I found I run a large number jobs with spark-sql at the same

[jira] [Updated] (SPARK-16870) add "spark.sql.broadcastTimeout" into docs/sql-programming-guide.md to help people to how to fix this timeout error when it happenned

2016-08-03 Thread Liang Ke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Ke updated SPARK-16870: - Description: here my workload and what I found I run a large number jobs with spark-sql at the same

[jira] [Commented] (SPARK-16870) add "spark.sql.broadcastTimeout" into docs/sql-programming-guide.md to help people to how to fix this timeout error when it happenned

2016-08-03 Thread Liang Ke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406834#comment-15406834 ] Liang Ke commented on SPARK-16870: -- thx :) I have update it > add "spark.sql.broadcastTimeout" into

[jira] [Updated] (SPARK-16870) add "spark.sql.broadcastTimeout" into docs/sql-programming-guide.md to help people to how to fix this timeout error when it happenned

2016-08-03 Thread Liang Ke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Ke updated SPARK-16870: - Description: here my workload and what I found I run a large number jobs with spark-sql at the same

[jira] [Assigned] (SPARK-16884) Move DataSourceScanExec out of ExistingRDD.scala file

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16884: Assignee: (was: Apache Spark) > Move DataSourceScanExec out of ExistingRDD.scala file

[jira] [Commented] (SPARK-16884) Move DataSourceScanExec out of ExistingRDD.scala file

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406779#comment-15406779 ] Apache Spark commented on SPARK-16884: -- User 'ericl' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16884) Move DataSourceScanExec out of ExistingRDD.scala file

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16884: Assignee: Apache Spark > Move DataSourceScanExec out of ExistingRDD.scala file >

[jira] [Updated] (SPARK-16884) Move DataSourceScanExec out of ExistingRDD.scala file

2016-08-03 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang updated SPARK-16884: --- Issue Type: Improvement (was: Bug) > Move DataSourceScanExec out of ExistingRDD.scala file >

[jira] [Created] (SPARK-16884) Move DataSourceScanExec out of ExistingRDD.scala file

2016-08-03 Thread Eric Liang (JIRA)
Eric Liang created SPARK-16884: -- Summary: Move DataSourceScanExec out of ExistingRDD.scala file Key: SPARK-16884 URL: https://issues.apache.org/jira/browse/SPARK-16884 Project: Spark Issue

[jira] [Created] (SPARK-16883) SQL decimal type is not properly cast to number when collecting SparkDataFrame

2016-08-03 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-16883: -- Summary: SQL decimal type is not properly cast to number when collecting SparkDataFrame Key: SPARK-16883 URL: https://issues.apache.org/jira/browse/SPARK-16883

[jira] [Commented] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-08-03 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406754#comment-15406754 ] Sylvain Zimmer commented on SPARK-16700: The verifySchema flag works great, and the {{dict}}

[jira] [Comment Edited] (SPARK-13710) Spark shell shows ERROR when launching on Windows

2016-08-03 Thread Arsen Vladimirskiy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406744#comment-15406744 ] Arsen Vladimirskiy edited comment on SPARK-13710 at 8/3/16 10:38 PM: -

[jira] [Commented] (SPARK-13710) Spark shell shows ERROR when launching on Windows

2016-08-03 Thread Arsen Vladimirskiy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406744#comment-15406744 ] Arsen Vladimirskiy commented on SPARK-13710: I noticed that when using the binary package

[jira] [Comment Edited] (SPARK-13710) Spark shell shows ERROR when launching on Windows

2016-08-03 Thread Arsen Vladimirskiy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406744#comment-15406744 ] Arsen Vladimirskiy edited comment on SPARK-13710 at 8/3/16 10:38 PM: -

[jira] [Resolved] (SPARK-15344) Unable to set default log level for PySpark

2016-08-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-15344. Resolution: Not A Problem See Felix's reply above. > Unable to set default log level for

[jira] [Commented] (SPARK-16455) Add a new hook in CoarseGrainedSchedulerBackend in order to stop scheduling new tasks when cluster is restarting

2016-08-03 Thread YangyangLiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406672#comment-15406672 ] YangyangLiu commented on SPARK-16455: - Ok. I get your point. I will wait for few days. If no one

[jira] [Commented] (SPARK-16796) Visible passwords on Spark environment page

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406571#comment-15406571 ] Apache Spark commented on SPARK-16796: -- User 'Devian-ua' has created a pull request for this issue:

[jira] [Updated] (SPARK-14204) [SQL] Failure to register URL-derived JDBC driver on executors in cluster mode

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14204: -- Fix Version/s: 2.1.0 2.0.1 > [SQL] Failure to register URL-derived JDBC driver on

[jira] [Closed] (SPARK-16857) CrossValidator and KMeans throws IllegalArgumentException

2016-08-03 Thread Ryan Claussen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Claussen closed SPARK-16857. - Usage error. > CrossValidator and KMeans throws IllegalArgumentException >

[jira] [Updated] (SPARK-16646) LEAST doesn't accept numeric arguments with different data types

2016-08-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16646: - Assignee: Wenchen Fan (was: Hyukjin Kwon) > LEAST doesn't accept numeric arguments with different data

[jira] [Resolved] (SPARK-16735) Fail to create a map contains decimal type with literals having different inferred precessions and scales

2016-08-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16735. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull request

[jira] [Updated] (SPARK-16735) Fail to create a map contains decimal type with literals having different inferred precessions and scales

2016-08-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16735: - Assignee: Wenchen Fan > Fail to create a map contains decimal type with literals having different >

[jira] [Resolved] (SPARK-16646) LEAST doesn't accept numeric arguments with different data types

2016-08-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16646. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull request

[jira] [Comment Edited] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-03 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406430#comment-15406430 ] Michael Allman edited comment on SPARK-16320 at 8/3/16 7:02 PM: Thank you

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-03 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406430#comment-15406430 ] Michael Allman commented on SPARK-16320: Thank you for the new benchmarks, [~maver1ck]. The

[jira] [Comment Edited] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405576#comment-15405576 ] Maciej Bryński edited comment on SPARK-16320 at 8/3/16 7:01 PM: You're

[jira] [Created] (SPARK-16882) Failures in JobGenerator Thread are Swallowed, Job Does Not Fail

2016-08-03 Thread Brian Schrameck (JIRA)
Brian Schrameck created SPARK-16882: --- Summary: Failures in JobGenerator Thread are Swallowed, Job Does Not Fail Key: SPARK-16882 URL: https://issues.apache.org/jira/browse/SPARK-16882 Project:

[jira] [Comment Edited] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405576#comment-15405576 ] Maciej Bryński edited comment on SPARK-16320 at 8/3/16 6:50 PM: You're

[jira] [Commented] (SPARK-16455) Add a new hook in CoarseGrainedSchedulerBackend in order to stop scheduling new tasks when cluster is restarting

2016-08-03 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406375#comment-15406375 ] Kay Ousterhout commented on SPARK-16455: Given that this feature is only needed for an internal

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406372#comment-15406372 ] Maciej Bryński commented on SPARK-16320: Yes. I also added Spark 1.6 with G1GC. Have in mind that

[jira] [Commented] (SPARK-16578) Configurable hostname for RBackend

2016-08-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406361#comment-15406361 ] Joseph K. Bradley commented on SPARK-16578: --- [~junyangq] has done some work on this. [~wm624]

[jira] [Commented] (SPARK-16714) Fail to create a decimal arrays with literals having different inferred precessions and scales

2016-08-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406351#comment-15406351 ] Yin Huai commented on SPARK-16714: -- [~petermaxlee] Can you create a new jira for your pr? I just merged

[jira] [Resolved] (SPARK-16596) Refactor DataSourceScanExec to do partition discovery at execution instead of planning time

2016-08-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16596. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14241

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-08-03 Thread Thomas Sebastian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406348#comment-15406348 ] Thomas Sebastian commented on SPARK-16753: -- [~jurriaanpruis]try a repartition before your

[jira] [Resolved] (SPARK-16714) Fail to create a decimal arrays with literals having different inferred precessions and scales

2016-08-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16714. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull request

[jira] [Updated] (SPARK-16714) Fail to create a decimal arrays with literals having different inferred precessions and scales

2016-08-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16714: - Assignee: Wenchen Fan > Fail to create a decimal arrays with literals having different inferred >

[jira] [Commented] (SPARK-16827) Query with Join produces excessive amount of shuffle data

2016-08-03 Thread Thomas Sebastian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406328#comment-15406328 ] Thomas Sebastian commented on SPARK-16827: -- I understand what you have mentioned is w.r.t to the

[jira] [Updated] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-08-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16610: - Target Version/s: 2.0.1, 2.1.0 > When writing ORC files, orc.compress should not be overridden if users

[jira] [Commented] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-08-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406329#comment-15406329 ] Yin Huai commented on SPARK-16610: -- It is a ORC setting. It should be accepted by the dataframe option

[jira] [Updated] (SPARK-16881) Migrate Mesos configs to use ConfigEntry

2016-08-03 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-16881: Description: https://github.com/apache/spark/pull/14414#discussion_r73032190 We'd like to

[jira] [Resolved] (SPARK-16857) CrossValidator and KMeans throws IllegalArgumentException

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16857. --- Resolution: Not A Problem The right way to do this is to evaluate based on the intra-cluster

[jira] [Commented] (SPARK-16881) Migrate Mesos configs to use ConfigEntry

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406312#comment-15406312 ] Sean Owen commented on SPARK-16881: --- (Worth summarizing the change or just at least copying the two

[jira] [Commented] (SPARK-16857) CrossValidator and KMeans throws IllegalArgumentException

2016-08-03 Thread Ryan Claussen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406311#comment-15406311 ] Ryan Claussen commented on SPARK-16857: --- Basically I want to use the {code}Validator{code} as a way

[jira] [Closed] (SPARK-16841) Improves the row level metrics performance when reading Parquet table

2016-08-03 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Zhong closed SPARK-16841. -- Resolution: Not A Problem > Improves the row level metrics performance when reading Parquet table >

[jira] [Comment Edited] (SPARK-16841) Improves the row level metrics performance when reading Parquet table

2016-08-03 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406306#comment-15406306 ] Sean Zhong edited comment on SPARK-16841 at 8/3/16 5:54 PM: This jira is

[jira] [Commented] (SPARK-16841) Improves the row level metrics performance when reading Parquet table

2016-08-03 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406306#comment-15406306 ] Sean Zhong commented on SPARK-16841: This PR is created after analyzing the performance impact of

[jira] [Comment Edited] (SPARK-16841) Improves the row level metrics performance when reading Parquet table

2016-08-03 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406306#comment-15406306 ] Sean Zhong edited comment on SPARK-16841 at 8/3/16 5:54 PM: This jira is

[jira] [Created] (SPARK-16881) Migrate Mesos configs to use ConfigEntry

2016-08-03 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-16881: --- Summary: Migrate Mesos configs to use ConfigEntry Key: SPARK-16881 URL: https://issues.apache.org/jira/browse/SPARK-16881 Project: Spark Issue Type:

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406245#comment-15406245 ] Sean Owen commented on SPARK-6305: -- Mostly to silence noisy libraries manually (i.e. still taking effect

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2016-08-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406207#comment-15406207 ] Mikael Ståldal commented on SPARK-6305: --- Yes, if you need to set logging level, then you need a

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2016-08-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406191#comment-15406191 ] Sean Owen commented on SPARK-6305: -- How would you set logging levels in slf4j though? I thought it didn't

[jira] [Assigned] (SPARK-16880) Improve ANN training, add training data persist if needed

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16880: Assignee: Apache Spark > Improve ANN training, add training data persist if needed >

[jira] [Assigned] (SPARK-16880) Improve ANN training, add training data persist if needed

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16880: Assignee: (was: Apache Spark) > Improve ANN training, add training data persist if

[jira] [Commented] (SPARK-16880) Improve ANN training, add training data persist if needed

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406156#comment-15406156 ] Apache Spark commented on SPARK-16880: -- User 'WeichenXu123' has created a pull request for this

[jira] [Updated] (SPARK-16880) Improve ANN training, add training data persist if needed

2016-08-03 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-16880: --- Component/s: MLlib ML > Improve ANN training, add training data persist if needed >

[jira] [Created] (SPARK-16880) Improve ANN training, add training data persist if needed

2016-08-03 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-16880: -- Summary: Improve ANN training, add training data persist if needed Key: SPARK-16880 URL: https://issues.apache.org/jira/browse/SPARK-16880 Project: Spark Issue

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2016-08-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406148#comment-15406148 ] Mikael Ståldal commented on SPARK-6305: --- Did you know that Log4j 2.x have it's own bridge from Log4j

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2016-08-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406142#comment-15406142 ] Mikael Ståldal commented on SPARK-6305: --- Which methods in Log4j 1.x are you missing counterparts in

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2016-08-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406140#comment-15406140 ] Mikael Ståldal commented on SPARK-6305: --- Would it be possible for Spark to depend on slf4j-api only

[jira] [Commented] (SPARK-16320) Spark 2.0 slower than 1.6 when querying nested columns

2016-08-03 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406125#comment-15406125 ] Michael Allman commented on SPARK-16320: Hi [~maver1ck]. These are excellent findings! I'm

[jira] [Assigned] (SPARK-16879) unify logical plans for CREATE TABLE and CTAS

2016-08-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16879: Assignee: Apache Spark (was: Wenchen Fan) > unify logical plans for CREATE TABLE and

  1   2   >