[jira] [Created] (SPARK-8784) Add python API for hex/unhex

2015-07-02 Thread Davies Liu (JIRA)
Davies Liu created SPARK-8784: - Summary: Add python API for hex/unhex Key: SPARK-8784 URL: https://issues.apache.org/jira/browse/SPARK-8784 Project: Spark Issue Type: Sub-task Component

[jira] [Assigned] (SPARK-8784) Add python API for hex/unhex

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8784: --- Assignee: Apache Spark (was: Davies Liu) > Add python API for hex/unhex > --

[jira] [Assigned] (SPARK-8784) Add python API for hex/unhex

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8784: --- Assignee: Davies Liu (was: Apache Spark) > Add python API for hex/unhex > --

[jira] [Commented] (SPARK-8784) Add python API for hex/unhex

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611575#comment-14611575 ] Apache Spark commented on SPARK-8784: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-8726) Wrong spark.executor.memory when using different EC2 master and worker machine types

2015-07-02 Thread Stefano Parmesan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611584#comment-14611584 ] Stefano Parmesan commented on SPARK-8726: - I've created a pull request for this is

[jira] [Commented] (SPARK-8632) Poor Python UDF performance because of RDD caching

2015-07-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611602#comment-14611602 ] Davies Liu commented on SPARK-8632: --- [~justin.uang] Sounds interesting, could you sendin

[jira] [Commented] (SPARK-8781) Pusblished POMs are no longer effective POMs

2015-07-02 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611600#comment-14611600 ] Konstantin Shaposhnikov commented on SPARK-8781: I believe this will affec

[jira] [Created] (SPARK-8785) Improve Parquet schema merging

2015-07-02 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-8785: -- Summary: Improve Parquet schema merging Key: SPARK-8785 URL: https://issues.apache.org/jira/browse/SPARK-8785 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-8785) Improve Parquet schema merging

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611605#comment-14611605 ] Apache Spark commented on SPARK-8785: - User 'viirya' has created a pull request for th

[jira] [Commented] (SPARK-8781) Pusblished POMs are no longer effective POMs

2015-07-02 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611606#comment-14611606 ] Konstantin Shaposhnikov commented on SPARK-8781: The original commit that

[jira] [Assigned] (SPARK-8785) Improve Parquet schema merging

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8785: --- Assignee: Apache Spark > Improve Parquet schema merging > -- > >

[jira] [Assigned] (SPARK-8785) Improve Parquet schema merging

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8785: --- Assignee: (was: Apache Spark) > Improve Parquet schema merging >

[jira] [Created] (SPARK-8786) Create a wrapper for BinaryType

2015-07-02 Thread Davies Liu (JIRA)
Davies Liu created SPARK-8786: - Summary: Create a wrapper for BinaryType Key: SPARK-8786 URL: https://issues.apache.org/jira/browse/SPARK-8786 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-8786) Create a wrapper for BinaryType

2015-07-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-8786: -- Description: The hashCode and equals() of Array[Byte] does check the bytes, we should create a wrapper (

[jira] [Commented] (SPARK-8706) Implement Pylint / Prospector checks for PySpark

2015-07-02 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611611#comment-14611611 ] Manoj Kumar commented on SPARK-8706: Sorry for sounding dumb, but the present code dow

[jira] [Commented] (SPARK-8781) Pusblished POMs are no longer effective POMs

2015-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611618#comment-14611618 ] Sean Owen commented on SPARK-8781: -- Right, I get all that. Yes that makes it clear what t

[jira] [Commented] (SPARK-8773) Throw type mismatch in check analysis for expressions with expected input types defined

2015-07-02 Thread Akhil Thatipamula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611623#comment-14611623 ] Akhil Thatipamula commented on SPARK-8773: -- [~rxin] aren't we checking that alrea

[jira] [Resolved] (SPARK-8729) Spark app unable to instantiate the classes using the reflection

2015-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8729. -- Resolution: Not A Problem > Spark app unable to instantiate the classes using the reflection > -

[jira] [Resolved] (SPARK-8691) Enable GZip for Web UI

2015-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8691. -- Resolution: Duplicate > Enable GZip for Web UI > -- > > Key: SPARK-8

[jira] [Created] (SPARK-8787) Change the parameter order of @deprecated in package object sql

2015-07-02 Thread Vinod KC (JIRA)
Vinod KC created SPARK-8787: --- Summary: Change the parameter order of @deprecated in package object sql Key: SPARK-8787 URL: https://issues.apache.org/jira/browse/SPARK-8787 Project: Spark Issue T

[jira] [Commented] (SPARK-8787) Change the parameter order of @deprecated in package object sql

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611650#comment-14611650 ] Apache Spark commented on SPARK-8787: - User 'vinodkc' has created a pull request for t

[jira] [Assigned] (SPARK-8787) Change the parameter order of @deprecated in package object sql

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8787: --- Assignee: (was: Apache Spark) > Change the parameter order of @deprecated in package obj

[jira] [Assigned] (SPARK-8787) Change the parameter order of @deprecated in package object sql

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8787: --- Assignee: Apache Spark > Change the parameter order of @deprecated in package object sql > -

[jira] [Created] (SPARK-8788) Java unit test for PCA transformer

2015-07-02 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-8788: -- Summary: Java unit test for PCA transformer Key: SPARK-8788 URL: https://issues.apache.org/jira/browse/SPARK-8788 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-8726) Wrong spark.executor.memory when using different EC2 master and worker machine types

2015-07-02 Thread Stefano Parmesan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefano Parmesan updated SPARK-8726: Description: _(this is a mirror of [MESOS-2985|https://issues.apache.org/jira/browse/MESOS-

[jira] [Updated] (SPARK-8726) Wrong spark.executor.memory when using different EC2 master and worker machine types

2015-07-02 Thread Stefano Parmesan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefano Parmesan updated SPARK-8726: Description: _(this is a mirror of [SPARK-8726|https://issues.apache.org/jira/browse/MESOS-

[jira] [Assigned] (SPARK-8788) Java unit test for PCA transformer

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8788: --- Assignee: Apache Spark > Java unit test for PCA transformer > ---

[jira] [Commented] (SPARK-8788) Java unit test for PCA transformer

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611666#comment-14611666 ] Apache Spark commented on SPARK-8788: - User 'yanboliang' has created a pull request fo

[jira] [Assigned] (SPARK-8788) Java unit test for PCA transformer

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8788: --- Assignee: (was: Apache Spark) > Java unit test for PCA transformer >

[jira] [Commented] (SPARK-8389) Expose KafkaRDDs offsetRange in Python

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611669#comment-14611669 ] Apache Spark commented on SPARK-8389: - User 'jerryshao' has created a pull request for

[jira] [Commented] (SPARK-8463) No suitable driver found for write.jdbc

2015-07-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611670#comment-14611670 ] Reynold Xin commented on SPARK-8463: [~mlety2] can you test this the patch created by

[jira] [Updated] (SPARK-8463) No suitable driver found for write.jdbc

2015-07-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8463: --- Shepherd: Reynold Xin Assignee: Liang-Chi Hsieh Target Version/s: 1.5.0, 1.4.2

[jira] [Commented] (SPARK-8684) Update R version in Spark EC2 AMI

2015-07-02 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611680#comment-14611680 ] Vincent Warmerdam commented on SPARK-8684: -- Mhm... I've tried multiple approaches

[jira] [Comment Edited] (SPARK-8684) Update R version in Spark EC2 AMI

2015-07-02 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611680#comment-14611680 ] Vincent Warmerdam edited comment on SPARK-8684 at 7/2/15 9:09 AM: --

[jira] [Comment Edited] (SPARK-8684) Update R version in Spark EC2 AMI

2015-07-02 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611680#comment-14611680 ] Vincent Warmerdam edited comment on SPARK-8684 at 7/2/15 9:10 AM: --

[jira] [Assigned] (SPARK-8244) string function: find_in_set

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8244: --- Assignee: Cheng Hao (was: Apache Spark) > string function: find_in_set > ---

[jira] [Commented] (SPARK-8244) string function: find_in_set

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611698#comment-14611698 ] Apache Spark commented on SPARK-8244: - User 'tarekauel' has created a pull request for

[jira] [Assigned] (SPARK-8244) string function: find_in_set

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8244: --- Assignee: Apache Spark (was: Cheng Hao) > string function: find_in_set > ---

[jira] [Commented] (SPARK-8596) Install and configure RStudio server on Spark EC2

2015-07-02 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611747#comment-14611747 ] Vincent Warmerdam commented on SPARK-8596: -- Cool, would love to hear your end of

[jira] [Updated] (SPARK-7401) Dot product and squared_distances should be vectorized in Vectors

2015-07-02 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Kumar updated SPARK-7401: --- Priority: Major (was: Minor) > Dot product and squared_distances should be vectorized in Vectors > --

[jira] [Commented] (SPARK-8616) SQLContext doesn't handle tricky column names when loading from JDBC

2015-07-02 Thread David Sabater (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611811#comment-14611811 ] David Sabater commented on SPARK-8616: -- I would assume the error here is the lack of

[jira] [Comment Edited] (SPARK-8616) SQLContext doesn't handle tricky column names when loading from JDBC

2015-07-02 Thread David Sabater (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611811#comment-14611811 ] David Sabater edited comment on SPARK-8616 at 7/2/15 11:39 AM: -

[jira] [Created] (SPARK-8790) BlockManager.reregister cause OOM

2015-07-02 Thread Patrick Liu (JIRA)
Patrick Liu created SPARK-8790: -- Summary: BlockManager.reregister cause OOM Key: SPARK-8790 URL: https://issues.apache.org/jira/browse/SPARK-8790 Project: Spark Issue Type: Bug Compone

[jira] [Created] (SPARK-8789) improve SQLQuerySuite resilience by dropping tables in setup

2015-07-02 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-8789: - Summary: improve SQLQuerySuite resilience by dropping tables in setup Key: SPARK-8789 URL: https://issues.apache.org/jira/browse/SPARK-8789 Project: Spark

[jira] [Updated] (SPARK-8790) BlockManager.reregister cause OOM

2015-07-02 Thread Patrick Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Liu updated SPARK-8790: --- Attachment: executor.log driver.log > BlockManager.reregister cause OOM >

[jira] [Updated] (SPARK-8790) BlockManager.reregister cause OOM

2015-07-02 Thread Patrick Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Liu updated SPARK-8790: --- Attachment: webui-executor.png > BlockManager.reregister cause OOM > -

[jira] [Updated] (SPARK-8790) BlockManager.reregister cause OOM

2015-07-02 Thread Patrick Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Liu updated SPARK-8790: --- Attachment: webui-slow-task.png > BlockManager.reregister cause OOM >

[jira] [Updated] (SPARK-8790) BlockManager.reregister cause OOM

2015-07-02 Thread Patrick Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Liu updated SPARK-8790: --- Description: We run SparkSQL 1.2.1 on Yarn. A SQL consists of 100 tasks, most them finish in < 10s, b

[jira] [Updated] (SPARK-8790) BlockManager.reregister cause OOM

2015-07-02 Thread Patrick Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Liu updated SPARK-8790: --- Description: We run SparkSQL 1.2.1 on Yarn. A SQL consists of 100 tasks, most them finish in < 10s, b

[jira] [Comment Edited] (SPARK-8596) Install and configure RStudio server on Spark EC2

2015-07-02 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611832#comment-14611832 ] Vincent Warmerdam edited comment on SPARK-8596 at 7/2/15 11:55 AM: -

[jira] [Commented] (SPARK-8596) Install and configure RStudio server on Spark EC2

2015-07-02 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611832#comment-14611832 ] Vincent Warmerdam commented on SPARK-8596: -- By the way, I now have scripts that d

[jira] [Updated] (SPARK-8790) BlockManager.reregister cause OOM

2015-07-02 Thread Patrick Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Liu updated SPARK-8790: --- Description: We run SparkSQL 1.2.1 on Yarn. A SQL consists of 100 tasks, most them finish in < 10s, b

[jira] [Commented] (SPARK-2319) Number of tasks on executors become negative after executor failures

2015-07-02 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611849#comment-14611849 ] KaiXinXIaoLei commented on SPARK-2319: -- using the lastest version (1.4),I also met th

[jira] [Updated] (SPARK-2319) Number of tasks on executors become negative after executor failures

2015-07-02 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-2319: - Attachment: active tasks.png Using 1.4, num active tasks become negative, and "Complete Tasks" is

[jira] [Comment Edited] (SPARK-2319) Number of tasks on executors become negative after executor failures

2015-07-02 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611856#comment-14611856 ] KaiXinXIaoLei edited comment on SPARK-2319 at 7/2/15 12:08 PM: -

[jira] [Commented] (SPARK-8789) improve SQLQuerySuite resilience by dropping tables in setup

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611871#comment-14611871 ] Apache Spark commented on SPARK-8789: - User 'steveloughran' has created a pull request

[jira] [Assigned] (SPARK-8789) improve SQLQuerySuite resilience by dropping tables in setup

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8789: --- Assignee: Apache Spark > improve SQLQuerySuite resilience by dropping tables in setup > -

[jira] [Assigned] (SPARK-8789) improve SQLQuerySuite resilience by dropping tables in setup

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8789: --- Assignee: (was: Apache Spark) > improve SQLQuerySuite resilience by dropping tables in se

[jira] [Updated] (SPARK-8787) Change the parameter order of @deprecated in package object sql

2015-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8787: - Assignee: Vinod KC > Change the parameter order of @deprecated in package object sql > --

[jira] [Resolved] (SPARK-8787) Change the parameter order of @deprecated in package object sql

2015-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8787. -- Resolution: Fixed Fix Version/s: 1.5.0 1.4.2 Issue resolved by pull request 71

[jira] [Updated] (SPARK-8746) Need to update download link for Hive 0.13.1 jars (HiveComparisonTest)

2015-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8746: - Assignee: Christian Kadner > Need to update download link for Hive 0.13.1 jars (HiveComparisonTest) >

[jira] [Resolved] (SPARK-8746) Need to update download link for Hive 0.13.1 jars (HiveComparisonTest)

2015-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8746. -- Resolution: Fixed Fix Version/s: 1.5.0 1.4.2 Issue resolved by pull request 71

[jira] [Updated] (SPARK-6833) Extend `addPackage` so that any given R file can be sourced in the worker before functions are run.

2015-07-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6833: - Assignee: Sun Rui > Extend `addPackage` so that any given R file can be sourced in the worker > before fu

[jira] [Created] (SPARK-8791) Make a better hashcode for InternalRow

2015-07-02 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-8791: Summary: Make a better hashcode for InternalRow Key: SPARK-8791 URL: https://issues.apache.org/jira/browse/SPARK-8791 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-8791) Make a better hashcode for InternalRow

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8791: --- Assignee: Apache Spark > Make a better hashcode for InternalRow > ---

[jira] [Commented] (SPARK-8791) Make a better hashcode for InternalRow

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611947#comment-14611947 ] Apache Spark commented on SPARK-8791: - User 'chenghao-intel' has created a pull reques

[jira] [Assigned] (SPARK-8791) Make a better hashcode for InternalRow

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8791: --- Assignee: (was: Apache Spark) > Make a better hashcode for InternalRow >

[jira] [Commented] (SPARK-5945) Spark should not retry a stage infinitely on a FetchFailedException

2015-07-02 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611960#comment-14611960 ] Daniel Darabos commented on SPARK-5945: --- At the moment we have a ton of these infini

[jira] [Created] (SPARK-8792) Add Python API for PCA transformer

2015-07-02 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-8792: -- Summary: Add Python API for PCA transformer Key: SPARK-8792 URL: https://issues.apache.org/jira/browse/SPARK-8792 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-8792) Add Python API for PCA transformer

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612002#comment-14612002 ] Apache Spark commented on SPARK-8792: - User 'yanboliang' has created a pull request fo

[jira] [Assigned] (SPARK-8792) Add Python API for PCA transformer

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8792: --- Assignee: Apache Spark > Add Python API for PCA transformer > ---

[jira] [Assigned] (SPARK-8792) Add Python API for PCA transformer

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8792: --- Assignee: (was: Apache Spark) > Add Python API for PCA transformer >

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

2015-07-02 Thread Eron Wright (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612015#comment-14612015 ] Eron Wright commented on SPARK-7768: - Will this issue cover making VectorUDT public?

[jira] [Updated] (SPARK-8793) error/warning with pyspark WholeTextFiles.first

2015-07-02 Thread Diana Carroll (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diana Carroll updated SPARK-8793: - Attachment: wholefilesbug.txt > error/warning with pyspark WholeTextFiles.first >

[jira] [Created] (SPARK-8793) error/warning with pyspark WholeTextFiles.first

2015-07-02 Thread Diana Carroll (JIRA)
Diana Carroll created SPARK-8793: Summary: error/warning with pyspark WholeTextFiles.first Key: SPARK-8793 URL: https://issues.apache.org/jira/browse/SPARK-8793 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-7483) [MLLib] Using Kryo with FPGrowth fails with an exception

2015-07-02 Thread Tomasz Bartczak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612060#comment-14612060 ] Tomasz Bartczak commented on SPARK-7483: this solves the problem for me as well.

[jira] [Created] (SPARK-8794) Column pruning isn't applied to certain transformed DataFrames

2015-07-02 Thread Eron Wright (JIRA)
Eron Wright created SPARK-8794: --- Summary: Column pruning isn't applied to certain transformed DataFrames Key: SPARK-8794 URL: https://issues.apache.org/jira/browse/SPARK-8794 Project: Spark I

[jira] [Commented] (SPARK-8293) Add high-level java docs to important YARN classes

2015-07-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-8293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612103#comment-14612103 ] Zoltán Zvara commented on SPARK-8293: - You can assign this issue to me. > Add high-le

[jira] [Commented] (SPARK-8294) Break down large methods in YARN code

2015-07-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-8294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612105#comment-14612105 ] Zoltán Zvara commented on SPARK-8294: - You can assign this to me. > Break down large

[jira] [Commented] (SPARK-8064) Upgrade Hive to 1.2

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612123#comment-14612123 ] Apache Spark commented on SPARK-8064: - User 'steveloughran' has created a pull request

[jira] [Created] (SPARK-8795) pySpark wholeTextFiles error when mapping string

2015-07-02 Thread Diana Carroll (JIRA)
Diana Carroll created SPARK-8795: Summary: pySpark wholeTextFiles error when mapping string Key: SPARK-8795 URL: https://issues.apache.org/jira/browse/SPARK-8795 Project: Spark Issue Type: Bu

[jira] [Updated] (SPARK-8795) pySpark wholeTextFiles error when mapping string

2015-07-02 Thread Diana Carroll (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diana Carroll updated SPARK-8795: - Attachment: wholefilesbug.txt > pySpark wholeTextFiles error when mapping string > ---

[jira] [Commented] (SPARK-8768) SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in Akka Protobuf

2015-07-02 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612173#comment-14612173 ] Shixiong Zhu commented on SPARK-8768: - [~joshrosen] is it the maven issue: https://gi

[jira] [Created] (SPARK-8796) clean closure at InsertIntoHiveTable and remove transient

2015-07-02 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-8796: -- Summary: clean closure at InsertIntoHiveTable and remove transient Key: SPARK-8796 URL: https://issues.apache.org/jira/browse/SPARK-8796 Project: Spark Issue Typ

[jira] [Commented] (SPARK-8596) Install and configure RStudio server on Spark EC2

2015-07-02 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612178#comment-14612178 ] Shivaram Venkataraman commented on SPARK-8596: -- Thanks [~cantdutchthis] -- To

[jira] [Commented] (SPARK-8684) Update R version in Spark EC2 AMI

2015-07-02 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612193#comment-14612193 ] Shivaram Venkataraman commented on SPARK-8684: -- Its unfortunate that the AWS

[jira] [Resolved] (SPARK-8690) Add a setting to disable SparkSQL parquet schema merge by using datasource API

2015-07-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-8690. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7070 [https://github.com/

[jira] [Resolved] (SPARK-8647) Potential issues with the constant hashCode

2015-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-8647. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7146 [https://githu

[jira] [Resolved] (SPARK-8758) Add Python user guide for PowerIterationClustering

2015-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-8758. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7155 [https://githu

[jira] [Assigned] (SPARK-8796) clean closure at InsertIntoHiveTable and remove transient

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8796: --- Assignee: Apache Spark > clean closure at InsertIntoHiveTable and remove transient >

[jira] [Commented] (SPARK-8796) clean closure at InsertIntoHiveTable and remove transient

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612180#comment-14612180 ] Apache Spark commented on SPARK-8796: - User 'cloud-fan' has created a pull request for

[jira] [Updated] (SPARK-8758) Add Python user guide for PowerIterationClustering

2015-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-8758: - Assignee: Yanbo Liang > Add Python user guide for PowerIterationClustering > -

[jira] [Assigned] (SPARK-8796) clean closure at InsertIntoHiveTable and remove transient

2015-07-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8796: --- Assignee: (was: Apache Spark) > clean closure at InsertIntoHiveTable and remove transient

[jira] [Resolved] (SPARK-8223) math function: shiftleft

2015-07-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8223. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7178 [https://github.com/

[jira] [Resolved] (SPARK-8224) math function: shiftright

2015-07-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8224. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7178 [https://github.com/

[jira] [Updated] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8781: - Summary: Published POMs are no longer effective POMs (was: Pusblished POMs are no longer effective POMs)

[jira] [Resolved] (SPARK-8747) fix EqualNullSafe for binary type

2015-07-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8747. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7143 [https://github.com/

[jira] [Updated] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8781: - Assignee: Andrew Or > Published POMs are no longer effective POMs > --

[jira] [Commented] (SPARK-8781) Published POMs are no longer effective POMs

2015-07-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612220#comment-14612220 ] Andrew Or commented on SPARK-8781: -- It was needed because of a bug in the maven shade plu

[jira] [Issue Comment Deleted] (SPARK-8294) Break down large methods in YARN code

2015-07-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-8294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltán Zvara updated SPARK-8294: Comment: was deleted (was: You can assign this to me.) > Break down large methods in YARN code > --

  1   2   3   >