[jira] [Updated] (SPARK-8628) Race condition in AbstractSparkSQLParser.parse

2015-07-01 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8628: Assignee: Vinod KC Race condition in AbstractSparkSQLParser.parse

[jira] [Updated] (SPARK-8308) add missing save load for python doc example and tune down MatrixFactorization iterations

2015-07-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8308: - Assignee: yuhao yang add missing save load for python doc example and tune down

[jira] [Commented] (SPARK-6101) Create a SparkSQL DataSource API implementation for DynamoDB

2015-07-01 Thread Murtaza Kanchwala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610799#comment-14610799 ] Murtaza Kanchwala commented on SPARK-6101: -- No, It is not a map function. For now

[jira] [Created] (SPARK-8766) DataFrame Python API should work with column which has non-ascii character in it

2015-07-01 Thread Davies Liu (JIRA)
Davies Liu created SPARK-8766: - Summary: DataFrame Python API should work with column which has non-ascii character in it Key: SPARK-8766 URL: https://issues.apache.org/jira/browse/SPARK-8766 Project:

[jira] [Updated] (SPARK-8647) Potential issues with the constant hashCode

2015-07-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-8647: - Assignee: Alok Singh Target Version/s: 1.5.0 Potential issues with the constant

[jira] [Resolved] (SPARK-8763) executing run-tests.py with Python 2.6 fails with absence of subprocess.check_output function

2015-07-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8763. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7161

[jira] [Updated] (SPARK-5427) Add support for floor function in Spark SQL

2015-07-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-5427: -- Description: floor() function is supported in Hive SQL. This issue is to add floor() function to Spark SQL.

[jira] [Updated] (SPARK-7714) SparkR tests should use more specific expectations than expect_true

2015-07-01 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-7714: - Assignee: Sun Rui SparkR tests should use more specific expectations than

[jira] [Resolved] (SPARK-7714) SparkR tests should use more specific expectations than expect_true

2015-07-01 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-7714. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request

[jira] [Resolved] (SPARK-8621) crosstab exception when one of the value is empty

2015-07-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8621. Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 1.4.2

[jira] [Updated] (SPARK-3071) Increase default driver memory

2015-07-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3071: - Affects Version/s: 1.4.2 Target Version/s: 1.5.0 Increase default driver memory

[jira] [Updated] (SPARK-3071) Increase default driver memory

2015-07-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3071: - Assignee: Ilya Ganelin Increase default driver memory --

[jira] [Updated] (SPARK-8744) StringIndexerModel should have public constructor

2015-07-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8744: - Description: It would be helpful to allow users to pass a pre-computed index to create an

[jira] [Updated] (SPARK-8072) Better AnalysisException for writing DataFrame with identically named columns

2015-07-01 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8072: Shepherd: Michael Armbrust Better AnalysisException for writing DataFrame with identically

[jira] [Resolved] (SPARK-7938) Use errorprone in Spark

2015-07-01 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-7938. --- Resolution: Won't Fix Use errorprone in Spark --- Key:

[jira] [Commented] (SPARK-8744) StringIndexerModel should have public constructor

2015-07-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610722#comment-14610722 ] Joseph K. Bradley commented on SPARK-8744: -- Good point, I'll link a JIRA for

[jira] [Resolved] (SPARK-6263) Python MLlib API missing items: Utils

2015-07-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-6263. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 5707

[jira] [Resolved] (SPARK-8308) add missing save load for python doc example and tune down MatrixFactorization iterations

2015-07-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-8308. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 6760

[jira] [Created] (SPARK-8765) Flaky PySpark PowerIterationClustering test

2015-07-01 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-8765: Summary: Flaky PySpark PowerIterationClustering test Key: SPARK-8765 URL: https://issues.apache.org/jira/browse/SPARK-8765 Project: Spark Issue

[jira] [Assigned] (SPARK-8765) Flaky PySpark PowerIterationClustering test

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8765: --- Assignee: (was: Apache Spark) Flaky PySpark PowerIterationClustering test

[jira] [Commented] (SPARK-8765) Flaky PySpark PowerIterationClustering test

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610783#comment-14610783 ] Apache Spark commented on SPARK-8765: - User 'jkbradley' has created a pull request for

[jira] [Assigned] (SPARK-8765) Flaky PySpark PowerIterationClustering test

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8765: --- Assignee: Apache Spark Flaky PySpark PowerIterationClustering test

[jira] [Updated] (SPARK-8765) Flaky PySpark PowerIterationClustering test

2015-07-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-8765: - Assignee: Yanbo Liang Flaky PySpark PowerIterationClustering test

[jira] [Updated] (SPARK-8751) Check missing and add user guide for MLlib Python API

2015-07-01 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-8751: --- Description: Some MLlib algorithm missing user guide for Python, we need to check and add them. The

[jira] [Created] (SPARK-8761) Master.removeApplication is not thread-safe but is called from multiple threads

2015-07-01 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-8761: --- Summary: Master.removeApplication is not thread-safe but is called from multiple threads Key: SPARK-8761 URL: https://issues.apache.org/jira/browse/SPARK-8761 Project:

[jira] [Assigned] (SPARK-8755) Streaming application from checkpoint will fail to load in security mode.

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8755: --- Assignee: (was: Apache Spark) Streaming application from checkpoint will fail to load

[jira] [Closed] (SPARK-8291) Add parse functionality to LabeledPoint in PySpark

2015-07-01 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Kumar closed SPARK-8291. -- Resolution: Won't Fix Add parse functionality to LabeledPoint in PySpark

[jira] [Commented] (SPARK-4557) Spark Streaming' foreachRDD method should accept a VoidFunction..., not a Function..., Void

2015-07-01 Thread Alexis Seigneurin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610376#comment-14610376 ] Alexis Seigneurin commented on SPARK-4557: -- Yes, but the problem is not compiling

[jira] [Commented] (SPARK-4557) Spark Streaming' foreachRDD method should accept a VoidFunction..., not a Function..., Void

2015-07-01 Thread somil deshmukh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610334#comment-14610334 ] somil deshmukh commented on SPARK-4557: --- In JavaDStreamLike.scala ,I have replace

[jira] [Assigned] (SPARK-8755) Streaming application from checkpoint will fail to load in security mode.

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8755: --- Assignee: Apache Spark Streaming application from checkpoint will fail to load in security

[jira] [Commented] (SPARK-8755) Streaming application from checkpoint will fail to load in security mode.

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610065#comment-14610065 ] Apache Spark commented on SPARK-8755: - User 'SaintBacchus' has created a pull request

[jira] [Commented] (SPARK-8734) Expose all Mesos DockerInfo options to Spark

2015-07-01 Thread Chris Heller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610289#comment-14610289 ] Chris Heller commented on SPARK-8734: - I've started work on this @

[jira] [Created] (SPARK-8762) Maven build fails if the project is in a symlinked folder

2015-07-01 Thread Roman Zenka (JIRA)
Roman Zenka created SPARK-8762: -- Summary: Maven build fails if the project is in a symlinked folder Key: SPARK-8762 URL: https://issues.apache.org/jira/browse/SPARK-8762 Project: Spark Issue

[jira] [Commented] (SPARK-6602) Replace direct use of Akka with Spark RPC interface

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610274#comment-14610274 ] Apache Spark commented on SPARK-6602: - User 'zsxwing' has created a pull request for

[jira] [Assigned] (SPARK-8756) Keep cached information and avoid re-calculating footers in ParquetRelation2

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8756: --- Assignee: Apache Spark Keep cached information and avoid re-calculating footers in

[jira] [Created] (SPARK-8756) Keep cached information and avoid re-calculating footers in ParquetRelation2

2015-07-01 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-8756: -- Summary: Keep cached information and avoid re-calculating footers in ParquetRelation2 Key: SPARK-8756 URL: https://issues.apache.org/jira/browse/SPARK-8756

[jira] [Assigned] (SPARK-8754) YarnClientSchedulerBackend doesn't stop gracefully in failure conditions

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8754: --- Assignee: (was: Apache Spark) YarnClientSchedulerBackend doesn't stop gracefully in

[jira] [Commented] (SPARK-8758) Add Python user guide for PowerIterationClustering

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609917#comment-14609917 ] Apache Spark commented on SPARK-8758: - User 'yanboliang' has created a pull request

[jira] [Closed] (SPARK-8751) Check missing and add user guide for MLlib Python API

2015-07-01 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang closed SPARK-8751. -- Resolution: Duplicate Check missing and add user guide for MLlib Python API

[jira] [Assigned] (SPARK-8758) Add Python user guide for PowerIterationClustering

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8758: --- Assignee: Apache Spark Add Python user guide for PowerIterationClustering

[jira] [Assigned] (SPARK-8758) Add Python user guide for PowerIterationClustering

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8758: --- Assignee: (was: Apache Spark) Add Python user guide for PowerIterationClustering

[jira] [Created] (SPARK-8760) allow moving and symlinking binaries

2015-07-01 Thread Philipp Angerer (JIRA)
Philipp Angerer created SPARK-8760: -- Summary: allow moving and symlinking binaries Key: SPARK-8760 URL: https://issues.apache.org/jira/browse/SPARK-8760 Project: Spark Issue Type:

[jira] [Created] (SPARK-8757) Check missing and add user guide for MLlib Python API

2015-07-01 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-8757: -- Summary: Check missing and add user guide for MLlib Python API Key: SPARK-8757 URL: https://issues.apache.org/jira/browse/SPARK-8757 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-8759) add default eval to binary and unary expression according to default behavior of nullable

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8759: --- Assignee: Apache Spark add default eval to binary and unary expression according to default

[jira] [Resolved] (SPARK-8731) Beeline doesn't work with -e option when started in background

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8731. -- Resolution: Invalid So you mean this is a Hive issue? Beeline doesn't work with -e option when

[jira] [Updated] (SPARK-8731) Beeline doesn't work with -e option when started in background

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8731: - Component/s: SQL Beeline doesn't work with -e option when started in background

[jira] [Commented] (SPARK-8759) add default eval to binary and unary expression according to default behavior of nullable

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609973#comment-14609973 ] Apache Spark commented on SPARK-8759: - User 'cloud-fan' has created a pull request for

[jira] [Updated] (SPARK-8236) misc function: crc32

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8236: - Assignee: Tarek Auel misc function: crc32 Key: SPARK-8236

[jira] [Updated] (SPARK-8235) misc function: sha1 / sha

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8235: - Assignee: Tarek Auel misc function: sha1 / sha - Key:

[jira] [Updated] (SPARK-8535) PySpark : Can't create DataFrame from Pandas dataframe with no explicit column name

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8535: - Assignee: Yuri Saito PySpark : Can't create DataFrame from Pandas dataframe with no explicit column

[jira] [Updated] (SPARK-8590) add code gen for ExtractValue

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8590: - Assignee: Wenchen Fan add code gen for ExtractValue -

[jira] [Updated] (SPARK-8589) cleanup DateTimeUtils

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8589: - Assignee: Wenchen Fan cleanup DateTimeUtils - Key: SPARK-8589

[jira] [Created] (SPARK-8754) YarnClientSchedulerBackend doesn't stop gracefully in failure conditions

2015-07-01 Thread Devaraj K (JIRA)
Devaraj K created SPARK-8754: Summary: YarnClientSchedulerBackend doesn't stop gracefully in failure conditions Key: SPARK-8754 URL: https://issues.apache.org/jira/browse/SPARK-8754 Project: Spark

[jira] [Commented] (SPARK-8754) YarnClientSchedulerBackend doesn't stop gracefully in failure conditions

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609842#comment-14609842 ] Apache Spark commented on SPARK-8754: - User 'devaraj-kavali' has created a pull

[jira] [Assigned] (SPARK-8754) YarnClientSchedulerBackend doesn't stop gracefully in failure conditions

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8754: --- Assignee: Apache Spark YarnClientSchedulerBackend doesn't stop gracefully in failure

[jira] [Updated] (SPARK-8031) Version number written to Hive metastore is 0.13.1aa instead of 0.13.1a

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8031: - Assignee: Cheng Lian Version number written to Hive metastore is 0.13.1aa instead of 0.13.1a

[jira] [Commented] (SPARK-1503) Implement Nesterov's accelerated first-order method

2015-07-01 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610018#comment-14610018 ] Kai Sasaki commented on SPARK-1503: --- [~staple] [~josephkb] Thank you for pinging and

[jira] [Updated] (SPARK-8755) Streaming application from checkpoint will fail to load in security mode.

2015-07-01 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-8755: Description: If the user set *spark.yarn.principal* and *spark.yarn.keytab* , he does not need

[jira] [Updated] (SPARK-8723) improve code gen for divide and remainder

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8723: - Assignee: Wenchen Fan improve code gen for divide and remainder

[jira] [Updated] (SPARK-8615) sql programming guide recommends deprecated code

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8615: - Assignee: Tijo Thomas sql programming guide recommends deprecated code

[jira] [Updated] (SPARK-8727) Add missing python api

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8727: - Assignee: Tarek Auel Add missing python api -- Key: SPARK-8727

[jira] [Updated] (SPARK-8692) re-order the case statements that handling catalyst data types

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8692: - Assignee: Wenchen Fan re-order the case statements that handling catalyst data types

[jira] [Updated] (SPARK-8751) Check missing and add user guide for MLlib Python API

2015-07-01 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-8751: --- Description: Some MLlib algorithm missing user guide for Python, we need to check and add them. The

[jira] [Created] (SPARK-8758) Add Python user guide for PowerIterationClustering

2015-07-01 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-8758: -- Summary: Add Python user guide for PowerIterationClustering Key: SPARK-8758 URL: https://issues.apache.org/jira/browse/SPARK-8758 Project: Spark Issue Type:

[jira] [Created] (SPARK-8759) add default eval to binary and unary expression according to default behavior of nullable

2015-07-01 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-8759: -- Summary: add default eval to binary and unary expression according to default behavior of nullable Key: SPARK-8759 URL: https://issues.apache.org/jira/browse/SPARK-8759

[jira] [Created] (SPARK-8755) Streaming application from checkpoint will fail to load in security mode.

2015-07-01 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-8755: --- Summary: Streaming application from checkpoint will fail to load in security mode. Key: SPARK-8755 URL: https://issues.apache.org/jira/browse/SPARK-8755 Project: Spark

[jira] [Assigned] (SPARK-8756) Keep cached information and avoid re-calculating footers in ParquetRelation2

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8756: --- Assignee: (was: Apache Spark) Keep cached information and avoid re-calculating footers

[jira] [Commented] (SPARK-8756) Keep cached information and avoid re-calculating footers in ParquetRelation2

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609852#comment-14609852 ] Apache Spark commented on SPARK-8756: - User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-8759) add default eval to binary and unary expression according to default behavior of nullable

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8759: --- Assignee: (was: Apache Spark) add default eval to binary and unary expression according

[jira] [Updated] (SPARK-3258) Python API for streaming MLlib algorithms

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3258: - Assignee: Manoj Kumar Python API for streaming MLlib algorithms

[jira] [Updated] (SPARK-7810) rdd.py _load_from_socket cannot load data from jvm socket if ipv6 is used

2015-07-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7810: - Assignee: Ai He rdd.py _load_from_socket cannot load data from jvm socket if ipv6 is used

[jira] [Created] (SPARK-8763) executing run-tests.py with Python 2.6 fails with absence of subprocess.check_output function

2015-07-01 Thread Tomohiko K. (JIRA)
Tomohiko K. created SPARK-8763: -- Summary: executing run-tests.py with Python 2.6 fails with absence of subprocess.check_output function Key: SPARK-8763 URL: https://issues.apache.org/jira/browse/SPARK-8763

[jira] [Commented] (SPARK-8744) StringIndexerModel should have public constructor

2015-07-01 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610558#comment-14610558 ] yuhao yang commented on SPARK-8744: --- Just a reminder: There seems to be more jobs to do

[jira] [Commented] (SPARK-4557) Spark Streaming' foreachRDD method should accept a VoidFunction..., not a Function..., Void

2015-07-01 Thread somil deshmukh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610414#comment-14610414 ] somil deshmukh commented on SPARK-4557: --- Can u provide me some example of

[jira] [Resolved] (SPARK-8265) Add LinearDataGenerator to pyspark.mllib.utils

2015-07-01 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Kumar resolved SPARK-8265. Resolution: Fixed Fix Version/s: 1.5.0 Add LinearDataGenerator to pyspark.mllib.utils

[jira] [Assigned] (SPARK-8763) executing run-tests.py with Python 2.6 fails with absence of subprocess.check_output function

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8763: --- Assignee: Apache Spark executing run-tests.py with Python 2.6 fails with absence of

[jira] [Commented] (SPARK-8763) executing run-tests.py with Python 2.6 fails with absence of subprocess.check_output function

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610530#comment-14610530 ] Apache Spark commented on SPARK-8763: - User 'cocoatomo' has created a pull request for

[jira] [Assigned] (SPARK-8733) ML RDD.unpersist calls should use blocking = false

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8733: --- Assignee: (was: Apache Spark) ML RDD.unpersist calls should use blocking = false

[jira] [Commented] (SPARK-8733) ML RDD.unpersist calls should use blocking = false

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610408#comment-14610408 ] Apache Spark commented on SPARK-8733: - User 'ilganeli' has created a pull request for

[jira] [Assigned] (SPARK-8763) executing run-tests.py with Python 2.6 fails with absence of subprocess.check_output function

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8763: --- Assignee: (was: Apache Spark) executing run-tests.py with Python 2.6 fails with absence

[jira] [Commented] (SPARK-8596) Install and configure RStudio server on Spark EC2

2015-07-01 Thread Mark Stephenson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610555#comment-14610555 ] Mark Stephenson commented on SPARK-8596: [~cantdutchthis]: we have been getting

[jira] [Commented] (SPARK-4557) Spark Streaming' foreachRDD method should accept a VoidFunction..., not a Function..., Void

2015-07-01 Thread Alexis Seigneurin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610456#comment-14610456 ] Alexis Seigneurin commented on SPARK-4557: -- Here:

[jira] [Assigned] (SPARK-8733) ML RDD.unpersist calls should use blocking = false

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8733: --- Assignee: Apache Spark ML RDD.unpersist calls should use blocking = false

[jira] [Updated] (SPARK-8765) Flaky PySpark PowerIterationClustering test

2015-07-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-8765: - Shepherd: Xiangrui Meng Flaky PySpark PowerIterationClustering test

[jira] [Commented] (SPARK-5016) GaussianMixtureEM should distribute matrix inverse for large numFeatures, k

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610961#comment-14610961 ] Apache Spark commented on SPARK-5016: - User 'feynmanliang' has created a pull request

[jira] [Updated] (SPARK-8765) Flaky PySpark PowerIterationClustering test

2015-07-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-8765: - Labels: flaky-test (was: ) Flaky PySpark PowerIterationClustering test

[jira] [Resolved] (SPARK-8378) Add Spark Flume Python API

2015-07-01 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-8378. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 1.5.0 Add Spark Flume

[jira] [Created] (SPARK-8767) Abstractions for InputColParam, OutputColParam

2015-07-01 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-8767: Summary: Abstractions for InputColParam, OutputColParam Key: SPARK-8767 URL: https://issues.apache.org/jira/browse/SPARK-8767 Project: Spark Issue

[jira] [Commented] (SPARK-8677) Decimal divide operation throws ArithmeticException

2015-07-01 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610865#comment-14610865 ] Jihong MA commented on SPARK-8677: -- I am not sure if there is guideline for

[jira] [Assigned] (SPARK-8766) DataFrame Python API should work with column which has non-ascii character in it

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8766: --- Assignee: Apache Spark (was: Davies Liu) DataFrame Python API should work with column

[jira] [Commented] (SPARK-8766) DataFrame Python API should work with column which has non-ascii character in it

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610942#comment-14610942 ] Apache Spark commented on SPARK-8766: - User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-8766) DataFrame Python API should work with column which has non-ascii character in it

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8766: --- Assignee: Davies Liu (was: Apache Spark) DataFrame Python API should work with column

[jira] [Updated] (SPARK-7820) Java8-tests suite compile error under SBT

2015-07-01 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7820: -- Assignee: Saisai Shao Java8-tests suite compile error under SBT

[jira] [Resolved] (SPARK-7820) Java8-tests suite compile error under SBT

2015-07-01 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-7820. --- Resolution: Fixed Fix Version/s: 1.5.0 1.4.2 Issue resolved by pull request

[jira] [Commented] (SPARK-8703) Add CountVectorizer as a ml transformer to convert document to words count vector

2015-07-01 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610970#comment-14610970 ] Feynman Liang commented on SPARK-8703: -- Took a second pass over the code and I agree

[jira] [Commented] (SPARK-1564) Add JavaScript into Javadoc to turn ::Experimental:: and such into badges

2015-07-01 Thread Deron Eriksson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610979#comment-14610979 ] Deron Eriksson commented on SPARK-1564: --- I'm working on this one. I believe this is

[jira] [Created] (SPARK-8768) SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in Akka Protobuf

2015-07-01 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-8768: - Summary: SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in Akka Protobuf Key: SPARK-8768 URL: https://issues.apache.org/jira/browse/SPARK-8768

[jira] [Commented] (SPARK-8764) StringIndexer should take option to handle unseen values

2015-07-01 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611056#comment-14611056 ] holdenk commented on SPARK-8764: I could do this, I've got another PR with the

[jira] [Commented] (SPARK-8768) SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in Akka Protobuf

2015-07-01 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610988#comment-14610988 ] Josh Rosen commented on SPARK-8768: --- We didn't notice this earlier because the Master

[jira] [Assigned] (SPARK-1564) Add JavaScript into Javadoc to turn ::Experimental:: and such into badges

2015-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-1564: --- Assignee: Apache Spark (was: Andrew Or) Add JavaScript into Javadoc to turn

  1   2   3   >