[jira] [Updated] (SPARK-13787) Feature importances for decision trees in Python

2016-03-10 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-13787: --- Priority: Minor (was: Major) > Feature importances for decision trees in Python > --

[jira] [Resolved] (SPARK-13787) Feature importances for decision trees in Python

2016-03-10 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-13787. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11622 [https:/

[jira] [Updated] (SPARK-13787) Feature importances for decision trees in Python

2016-03-10 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-13787: --- Assignee: Seth Hendrickson > Feature importances for decision trees in Python > -

[jira] [Assigned] (SPARK-13814) Delete unnecessary imports in python examples files

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13814: Assignee: Apache Spark > Delete unnecessary imports in python examples files > ---

[jira] [Assigned] (SPARK-13814) Delete unnecessary imports in python examples files

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13814: Assignee: (was: Apache Spark) > Delete unnecessary imports in python examples files >

[jira] [Commented] (SPARK-13814) Delete unnecessary imports in python examples files

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190582#comment-15190582 ] Apache Spark commented on SPARK-13814: -- User 'zhengruifeng' has created a pull reque

[jira] [Closed] (SPARK-13813) Fix SparkR lint-r test errors

2016-03-10 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-13813. --- Resolution: Duplicate > Fix SparkR lint-r test errors > - > > Key

[jira] [Created] (SPARK-13814) Delete unnecessary imports in python examples files

2016-03-10 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-13814: Summary: Delete unnecessary imports in python examples files Key: SPARK-13814 URL: https://issues.apache.org/jira/browse/SPARK-13814 Project: Spark Issue Typ

[jira] [Resolved] (SPARK-13512) Add example and doc for ml.feature.MaxAbsScaler

2016-03-10 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-13512. Resolution: Fixed Fix Version/s: 2.0.0 > Add example and doc for ml.feature.MaxAbsSc

[jira] [Resolved] (SPARK-13294) Remove MiMa's dependency on spark-class / Spark assembly

2016-03-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-13294. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11178 [https://github.

[jira] [Created] (SPARK-13813) Fix SparkR lint-r test errors

2016-03-10 Thread Sun Rui (JIRA)
Sun Rui created SPARK-13813: --- Summary: Fix SparkR lint-r test errors Key: SPARK-13813 URL: https://issues.apache.org/jira/browse/SPARK-13813 Project: Spark Issue Type: Bug Components: Spa

[jira] [Created] (SPARK-13812) Fix SparkR lint-r test errors

2016-03-10 Thread Sun Rui (JIRA)
Sun Rui created SPARK-13812: --- Summary: Fix SparkR lint-r test errors Key: SPARK-13812 URL: https://issues.apache.org/jira/browse/SPARK-13812 Project: Spark Issue Type: Bug Components: Spa

[jira] [Resolved] (SPARK-13672) Add python examples of BisectingKMeans in ML and MLLIB

2016-03-10 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-13672. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11515 [https:/

[jira] [Updated] (SPARK-13800) Hive conf will be modified on multi-beeline connect to thriftserver

2016-03-10 Thread Weizhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weizhong updated SPARK-13800: - Description: {color:red}connect to Hive MetaStore service as we have set hive.metastore.uris in hive-sit

[jira] [Assigned] (SPARK-13811) No Push Down for Null Filtering of Compound Expressions Generated from Constraints

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13811: Assignee: Apache Spark > No Push Down for Null Filtering of Compound Expressions Generated

[jira] [Commented] (SPARK-13811) No Push Down for Null Filtering of Compound Expressions Generated from Constraints

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190541#comment-15190541 ] Apache Spark commented on SPARK-13811: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-13811) No Push Down for Null Filtering of Compound Expressions Generated from Constraints

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13811: Assignee: (was: Apache Spark) > No Push Down for Null Filtering of Compound Expression

[jira] [Created] (SPARK-13811) No Push Down for Null Filtering of Compound Expressions Generated from Constraints

2016-03-10 Thread Xiao Li (JIRA)
Xiao Li created SPARK-13811: --- Summary: No Push Down for Null Filtering of Compound Expressions Generated from Constraints Key: SPARK-13811 URL: https://issues.apache.org/jira/browse/SPARK-13811 Project: Spa

[jira] [Assigned] (SPARK-13809) State Store: A new framework for state management for computing Streaming Aggregates

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13809: Assignee: Tathagata Das (was: Apache Spark) > State Store: A new framework for state mana

[jira] [Commented] (SPARK-13809) State Store: A new framework for state management for computing Streaming Aggregates

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190529#comment-15190529 ] Apache Spark commented on SPARK-13809: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-13809) State Store: A new framework for state management for computing Streaming Aggregates

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13809: Assignee: Apache Spark (was: Tathagata Das) > State Store: A new framework for state mana

[jira] [Updated] (SPARK-13800) Hive conf will be modified on multi-beeline connect to thriftserver

2016-03-10 Thread Weizhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weizhong updated SPARK-13800: - Description: {color:red}connect to Hive MetaStore service as we have set hive.metastore.uris in hive-sit

[jira] [Resolved] (SPARK-13797) Eliminate Unnecessary Window

2016-03-10 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-13797. - Resolution: Fixed Fix Version/s: 2.0.0 > Eliminate Unnecessary Window > --

[jira] [Resolved] (SPARK-13732) Remove projectList from Windows

2016-03-10 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-13732. - Resolution: Fixed Fix Version/s: 2.0.0 > Remove projectList from Windows > ---

[jira] [Assigned] (SPARK-13658) BooleanSimplification rule is slow with large boolean expressions

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13658: Assignee: (was: Apache Spark) > BooleanSimplification rule is slow with large boolean

[jira] [Assigned] (SPARK-13658) BooleanSimplification rule is slow with large boolean expressions

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13658: Assignee: Apache Spark > BooleanSimplification rule is slow with large boolean expressions

[jira] [Commented] (SPARK-13658) BooleanSimplification rule is slow with large boolean expressions

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190480#comment-15190480 ] Apache Spark commented on SPARK-13658: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-13810) Add Port Configuration Suggestions on Bind Exceptions

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13810: Assignee: (was: Apache Spark) > Add Port Configuration Suggestions on Bind Exceptions

[jira] [Assigned] (SPARK-13810) Add Port Configuration Suggestions on Bind Exceptions

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13810: Assignee: Apache Spark > Add Port Configuration Suggestions on Bind Exceptions > -

[jira] [Commented] (SPARK-13810) Add Port Configuration Suggestions on Bind Exceptions

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190437#comment-15190437 ] Apache Spark commented on SPARK-13810: -- User 'bjornjon' has created a pull request f

[jira] [Created] (SPARK-13810) Add Port Configuration Suggestions on Bind Exceptions

2016-03-10 Thread Bjorn Jonsson (JIRA)
Bjorn Jonsson created SPARK-13810: - Summary: Add Port Configuration Suggestions on Bind Exceptions Key: SPARK-13810 URL: https://issues.apache.org/jira/browse/SPARK-13810 Project: Spark Issue

[jira] [Created] (SPARK-13809) State Store: A new framework for state management for computing Streaming Aggregates

2016-03-10 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-13809: - Summary: State Store: A new framework for state management for computing Streaming Aggregates Key: SPARK-13809 URL: https://issues.apache.org/jira/browse/SPARK-13809

[jira] [Commented] (SPARK-8360) Streaming DataFrames

2016-03-10 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190414#comment-15190414 ] Tathagata Das commented on SPARK-8360: -- This is still highly WIP, and not ready for e

[jira] [Commented] (SPARK-13618) Make Streaming web UI display rate-limit lines in the statistics graph

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190374#comment-15190374 ] Apache Spark commented on SPARK-13618: -- User 'lw-lin' has created a pull request for

[jira] [Updated] (SPARK-13800) Hive conf will be modified on multi-beeline connect to thriftserver

2016-03-10 Thread Weizhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weizhong updated SPARK-13800: - Description: {color:red}connect to Hive MetaStore service as we have set hive.metastore.uris in hive-sit

[jira] [Updated] (SPARK-13726) Spark 1.6.0 stopping working for HiveThriftServer2 and registerTempTable

2016-03-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-13726: - Target Version/s: (was: 1.6.2) > Spark 1.6.0 stopping working for HiveThriftServer2 and registerTempTab

[jira] [Commented] (SPARK-13726) Spark 1.6.0 stopping working for HiveThriftServer2 and registerTempTable

2016-03-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190340#comment-15190340 ] Yin Huai commented on SPARK-13726: -- ah, forgot to update this JIRA. I think the behavior

[jira] [Resolved] (SPARK-13726) Spark 1.6.0 stopping working for HiveThriftServer2 and registerTempTable

2016-03-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-13726. -- Resolution: Not A Problem > Spark 1.6.0 stopping working for HiveThriftServer2 and registerTempTable >

[jira] [Resolved] (SPARK-13656) Delete spark.sql.parquet.cacheMetadata

2016-03-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-13656. -- Resolution: Later Seems this flag stills controls the behavior of SqlNewHadoopRDD, let's revisit this

[jira] [Comment Edited] (SPARK-13656) Delete spark.sql.parquet.cacheMetadata

2016-03-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190314#comment-15190314 ] Yin Huai edited comment on SPARK-13656 at 3/11/16 1:44 AM: --- See

[jira] [Resolved] (SPARK-13389) SparkR support first/last with ignore NAs

2016-03-10 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-13389. --- Resolution: Fixed Assignee: Yanbo Liang Fix Version/s: 2.0.0

[jira] [Resolved] (SPARK-13789) Infer additional constraints from attribute equality

2016-03-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13789. - Resolution: Fixed Assignee: Sameer Agarwal Fix Version/s: 2.0.0 > Infer additiona

[jira] [Comment Edited] (SPARK-13804) Spark SQL's DataFrame.count() Major Divergent (Non-Linear) Performance Slowdown going from 4million rows to 16+ million rows

2016-03-10 Thread Michael Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189827#comment-15189827 ] Michael Nguyen edited comment on SPARK-13804 at 3/11/16 1:28 AM: --

[jira] [Updated] (SPARK-13804) Spark SQL's DataFrame.count() Major Divergent (Non-Linear) Performance Slowdown going from 4million rows to 16+ million rows

2016-03-10 Thread Michael Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Nguyen updated SPARK-13804: --- Environment: - 3 nodes Spark cluster: 1 master node and 2 slave nodes - Each node is an EC2 w

[jira] [Updated] (SPARK-13294) Remove MiMa's dependency on spark-class / Spark assembly

2016-03-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-13294: --- Description: We should remove {{dev/mima}}'s dependency on the Spark assembly was:As of SPARK-928

[jira] [Resolved] (SPARK-13327) colnames()<- allows invalid column names

2016-03-10 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-13327. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.2 Resolv

[jira] [Created] (SPARK-13808) Don't build assembly in dev/run-tests

2016-03-10 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-13808: -- Summary: Don't build assembly in dev/run-tests Key: SPARK-13808 URL: https://issues.apache.org/jira/browse/SPARK-13808 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-13327) colnames()<- allows invalid column names

2016-03-10 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-13327: -- Assignee: Oscar D. Lara Yejas > colnames()<- allows invalid column names >

[jira] [Assigned] (SPARK-13780) SQL "incremental" build in maven is broken

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13780: Assignee: Apache Spark > SQL "incremental" build in maven is broken >

[jira] [Assigned] (SPARK-13780) SQL "incremental" build in maven is broken

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13780: Assignee: (was: Apache Spark) > SQL "incremental" build in maven is broken > -

[jira] [Assigned] (SPARK-13807) De-duplicate `Python*Helper` instantiation code in PySpark streaming

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13807: Assignee: Josh Rosen (was: Apache Spark) > De-duplicate `Python*Helper` instantiation cod

[jira] [Updated] (SPARK-13294) Remove MiMa's dependency on spark-class / Spark assembly

2016-03-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-13294: --- Summary: Remove MiMa's dependency on spark-class / Spark assembly (was: Don't build assembly in dev/

[jira] [Assigned] (SPARK-13807) De-duplicate `Python*Helper` instantiation code in PySpark streaming

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13807: Assignee: Apache Spark (was: Josh Rosen) > De-duplicate `Python*Helper` instantiation cod

[jira] [Commented] (SPARK-13807) De-duplicate `Python*Helper` instantiation code in PySpark streaming

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190270#comment-15190270 ] Apache Spark commented on SPARK-13807: -- User 'JoshRosen' has created a pull request

[jira] [Commented] (SPARK-13780) SQL "incremental" build in maven is broken

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190269#comment-15190269 ] Apache Spark commented on SPARK-13780: -- User 'vanzin' has created a pull request for

[jira] [Created] (SPARK-13807) De-duplicate `Python*Helper` instantiation code in PySpark streaming

2016-03-10 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-13807: -- Summary: De-duplicate `Python*Helper` instantiation code in PySpark streaming Key: SPARK-13807 URL: https://issues.apache.org/jira/browse/SPARK-13807 Project: Spark

[jira] [Resolved] (SPARK-13604) Sync worker's state after registering with master

2016-03-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13604. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Sync worker's state

[jira] [Resolved] (SPARK-13244) Unify DataFrame and Dataset API

2016-03-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-13244. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11443 [https://github.com/

[jira] [Resolved] (SPARK-13751) Generate better code for Filter

2016-03-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13751. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11585 [https://github.

[jira] [Commented] (SPARK-13776) Web UI is not available after ./sbin/start-master.sh

2016-03-10 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190227#comment-15190227 ] Shixiong Zhu commented on SPARK-13776: -- I didn't count the number of selectors. [~er

[jira] [Updated] (SPARK-3854) Scala style: require spaces before `{`

2016-03-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3854: - Assignee: Dongjoon Hyun > Scala style: require spaces before `{` > --

[jira] [Resolved] (SPARK-3854) Scala style: require spaces before `{`

2016-03-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-3854. -- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Scala style: require s

[jira] [Updated] (SPARK-13806) SQL round() produces incorrect results for negative values

2016-03-10 Thread Mark Hamstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hamstra updated SPARK-13806: - Description: Round in catalyst/expressions/mathExpressions.scala appears to be untested with neg

[jira] [Created] (SPARK-13806) SQL round() produces incorrect results for negative values

2016-03-10 Thread Mark Hamstra (JIRA)
Mark Hamstra created SPARK-13806: Summary: SQL round() produces incorrect results for negative values Key: SPARK-13806 URL: https://issues.apache.org/jira/browse/SPARK-13806 Project: Spark Is

[jira] [Resolved] (SPARK-13696) Remove BlockStore interface to more cleanly reflect different memory and disk store responsibilities

2016-03-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13696. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove BlockStore interface to more cleanly reflect d

[jira] [Commented] (SPARK-10765) use new aggregate interface for hive UDAF

2016-03-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190120#comment-15190120 ] Yin Huai commented on SPARK-10765: -- [~dyross] I think we have never done partial aggrega

[jira] [Updated] (SPARK-13803) Standalone master does not balance cluster-mode drivers across workers

2016-03-10 Thread Brian Wongchaowart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Wongchaowart updated SPARK-13803: --- Component/s: Deploy > Standalone master does not balance cluster-mode drivers across

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-03-10 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190103#comment-15190103 ] Cody Koeninger commented on SPARK-12177: There are a lot of things I'm really not

[jira] [Commented] (SPARK-10931) PySpark ML Models should contain Param values

2016-03-10 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190093#comment-15190093 ] Timothy Hunter commented on SPARK-10931: Using python decorators, it is fairly ea

[jira] [Updated] (SPARK-13795) ClassCast Exception while attempting to show() a DataFrame

2016-03-10 Thread Ganesh Krishnan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ganesh Krishnan updated SPARK-13795: Description: DataFrame Schema (by printSchema() ) is as follows allDataJoined.printSchema(

[jira] [Closed] (SPARK-13788) CholeskyDecomposition has side effects!

2016-03-10 Thread Ehsan Mohyedin Kermani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ehsan Mohyedin Kermani closed SPARK-13788. -- Resolution: Not A Problem > CholeskyDecomposition has side effects! > -

[jira] [Commented] (SPARK-10765) use new aggregate interface for hive UDAF

2016-03-10 Thread David Ross (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190042#comment-15190042 ] David Ross commented on SPARK-10765: As noted in the change, this is a performance re

[jira] [Resolved] (SPARK-13790) Speed up ColumnVector's getDecimal

2016-03-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13790. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11624 [https://github.

[jira] [Commented] (SPARK-13796) Lock release errors occur frequently in executor logs

2016-03-10 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189957#comment-15189957 ] Nishkam Ravi commented on SPARK-13796: -- Would it make sense to change the condition

[jira] [Commented] (SPARK-13796) Lock release errors occur frequently in executor logs

2016-03-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189939#comment-15189939 ] Josh Rosen commented on SPARK-13796: This is likely to be a spurious warning due to t

[jira] [Commented] (SPARK-13776) Web UI is not available after ./sbin/start-master.sh

2016-03-10 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189936#comment-15189936 ] Shixiong Zhu commented on SPARK-13776: -- Looks I didn't detect the correct min thread

[jira] [Commented] (SPARK-13796) Lock release errors occur frequently in executor logs

2016-03-10 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189930#comment-15189930 ] Nishkam Ravi commented on SPARK-13796: -- [~joshrosen] Retested with latest master. FW

[jira] [Commented] (SPARK-13796) Lock release errors occur frequently in executor logs

2016-03-10 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189919#comment-15189919 ] Nishkam Ravi commented on SPARK-13796: -- Hey [~srowen] this doesn't seem to impact fu

[jira] [Commented] (SPARK-13776) Web UI is not available after ./sbin/start-master.sh

2016-03-10 Thread Erik O'Shaughnessy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189906#comment-15189906 ] Erik O'Shaughnessy commented on SPARK-13776: I have built and tested your fix

[jira] [Assigned] (SPARK-3854) Scala style: require spaces before `{`

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3854: --- Assignee: Apache Spark > Scala style: require spaces before `{` > ---

[jira] [Commented] (SPARK-3854) Scala style: require spaces before `{`

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189891#comment-15189891 ] Apache Spark commented on SPARK-3854: - User 'dongjoon-hyun' has created a pull request

[jira] [Assigned] (SPARK-3854) Scala style: require spaces before `{`

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3854: --- Assignee: (was: Apache Spark) > Scala style: require spaces before `{` >

[jira] [Commented] (SPARK-3854) Scala style: require spaces before `{`

2016-03-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189886#comment-15189886 ] Dongjoon Hyun commented on SPARK-3854: -- Hi, all. It seems to be a long lasting issue

[jira] [Updated] (SPARK-13759) Add IsNotNull constraints for expressions with an inequality

2016-03-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-13759: - Assignee: Sameer Agarwal > Add IsNotNull constraints for expressions with an inequality > ---

[jira] [Resolved] (SPARK-13759) Add IsNotNull constraints for expressions with an inequality

2016-03-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-13759. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11594 [https://github.com/

[jira] [Comment Edited] (SPARK-13804) org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 hangs intermittently

2016-03-10 Thread Michael Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189827#comment-15189827 ] Michael Nguyen edited comment on SPARK-13804 at 3/10/16 8:03 PM: --

[jira] [Comment Edited] (SPARK-13804) org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 hangs intermittently

2016-03-10 Thread Michael Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189827#comment-15189827 ] Michael Nguyen edited comment on SPARK-13804 at 3/10/16 8:02 PM: --

[jira] [Commented] (SPARK-13804) org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 hangs intermittently

2016-03-10 Thread Michael Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189827#comment-15189827 ] Michael Nguyen commented on SPARK-13804: HiveThriftServer2 is part of org.apache.

[jira] [Updated] (SPARK-13804) org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 hangs intermittently

2016-03-10 Thread Michael Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Nguyen updated SPARK-13804: --- Summary: org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 hangs intermittently (was:

[jira] [Assigned] (SPARK-13805) Direct consume ColumnVector in generated code when ColumnarBatch is used

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13805: Assignee: Apache Spark > Direct consume ColumnVector in generated code when ColumnarBatch

[jira] [Assigned] (SPARK-13805) Direct consume ColumnVector in generated code when ColumnarBatch is used

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13805: Assignee: (was: Apache Spark) > Direct consume ColumnVector in generated code when Col

[jira] [Commented] (SPARK-13805) Direct consume ColumnVector in generated code when ColumnarBatch is used

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189821#comment-15189821 ] Apache Spark commented on SPARK-13805: -- User 'kiszk' has created a pull request for

[jira] [Commented] (SPARK-13804) HiveThriftServer2 hangs intermittently

2016-03-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189792#comment-15189792 ] Sean Owen commented on SPARK-13804: --- Isn't this a Hive issue? > HiveThriftServer2 hang

[jira] [Resolved] (SPARK-13727) SparkConf.contains does not consider deprecated keys

2016-03-10 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-13727. Resolution: Fixed Assignee: Bo Meng Fix Version/s: 2.0.0 > SparkConf.contai

[jira] [Resolved] (SPARK-13794) Rename DataFrameWriter.stream DataFrameWriter.startStream

2016-03-10 Thread Ram Sriharsha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ram Sriharsha resolved SPARK-13794. --- Resolution: Fixed Fix Version/s: 2.0.0 > Rename DataFrameWriter.stream DataFrameWriter

[jira] [Created] (SPARK-13805) Direct consume ColumnVector in generated code when ColumnarBatch is used

2016-03-10 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-13805: Summary: Direct consume ColumnVector in generated code when ColumnarBatch is used Key: SPARK-13805 URL: https://issues.apache.org/jira/browse/SPARK-13805 Proj

[jira] [Updated] (SPARK-13795) ClassCast Exception while attempting to show() a DataFrame

2016-03-10 Thread Ram Sriharsha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ram Sriharsha updated SPARK-13795: -- Description: DataFrame Schema (by printSchema() ) is as follows allDataJoined.printSchema()

[jira] [Updated] (SPARK-13795) ClassCast Exception while attempting to show() a DataFrame

2016-03-10 Thread Ram Sriharsha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ram Sriharsha updated SPARK-13795: -- Description: DataFrame Schema (by printSchema() ) is as follows allDataJoined.printSchema()

[jira] [Assigned] (SPARK-13739) Predicate Push Down Through Window Operator

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13739: Assignee: (was: Apache Spark) > Predicate Push Down Through Window Operator >

[jira] [Commented] (SPARK-13739) Predicate Push Down Through Window Operator

2016-03-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189719#comment-15189719 ] Apache Spark commented on SPARK-13739: -- User 'gatorsmile' has created a pull request

  1   2   >