[jira] [Created] (SPARK-12524) Group by key in a pairrdd without any shuffle

2015-12-25 Thread Shushant Arora (JIRA)
Shushant Arora created SPARK-12524: -- Summary: Group by key in a pairrdd without any shuffle Key: SPARK-12524 URL: https://issues.apache.org/jira/browse/SPARK-12524 Project: Spark Issue

[jira] [Closed] (SPARK-12518) Problem in Spark deserialization with htsjdk BAMRecordCodec

2015-12-25 Thread Zhanpeng Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhanpeng Wu closed SPARK-12518. --- > Problem in Spark deserialization with htsjdk BAMRecordCodec >

[jira] [Resolved] (SPARK-12518) Problem in Spark deserialization with htsjdk BAMRecordCodec

2015-12-25 Thread Zhanpeng Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhanpeng Wu resolved SPARK-12518. - Resolution: Fixed > Problem in Spark deserialization with htsjdk BAMRecordCodec >

[jira] [Commented] (SPARK-12491) UDAF result differs in SQL if alias is used

2015-12-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071454#comment-15071454 ] Herman van Hovell commented on SPARK-12491: --- I cannot reproduce this issue on 1.5.2 and the

[jira] [Updated] (SPARK-12516) Properly handle NM failure situation for Spark on Yarn

2015-12-25 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-12516: Description: Failure of NodeManager will make all the executors belong to that NM exit silently.

[jira] [Commented] (SPARK-12521) DataFrame Partitions in java does not work

2015-12-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071432#comment-15071432 ] Herman van Hovell commented on SPARK-12521: --- The {{lowerBound}} and {{upperBound}} parameters

[jira] [Commented] (SPARK-12524) Group by key in a pairrdd without any shuffle

2015-12-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071473#comment-15071473 ] Herman van Hovell commented on SPARK-12524: --- I think you are looking for something similar to

[jira] [Created] (SPARK-12523) Support long-running of the Spark On HBase and hive meta store.

2015-12-25 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-12523: Summary: Support long-running of the Spark On HBase and hive meta store. Key: SPARK-12523 URL: https://issues.apache.org/jira/browse/SPARK-12523 Project: Spark

[jira] [Commented] (SPARK-12518) Problem in Spark deserialization with htsjdk BAMRecordCodec

2015-12-25 Thread Zhanpeng Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071419#comment-15071419 ] Zhanpeng Wu commented on SPARK-12518: - I have solved this issue. The problem is the BAMRecordCodec

[jira] [Comment Edited] (SPARK-12520) Python API dataframe join returns wrong results on outer join

2015-12-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071695#comment-15071695 ] Xiao Li edited comment on SPARK-12520 at 12/25/15 6:48 PM: --- The problem still

[jira] [Commented] (SPARK-12524) Group by key in a pairrdd without any shuffle

2015-12-25 Thread Shushant Arora (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071504#comment-15071504 ] Shushant Arora commented on SPARK-12524: Its not similar to bucketisation- Its for the scenaro

[jira] [Assigned] (SPARK-12385) Push projection into Join

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12385: Assignee: (was: Apache Spark) > Push projection into Join > -

[jira] [Assigned] (SPARK-12385) Push projection into Join

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12385: Assignee: Apache Spark > Push projection into Join > - > >

[jira] [Commented] (SPARK-12385) Push projection into Join

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071665#comment-15071665 ] Apache Spark commented on SPARK-12385: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Updated] (SPARK-12521) DataFrame Partitions in java does not work

2015-12-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-12521: Component/s: SQL > DataFrame Partitions in java does not work > --

[jira] [Comment Edited] (SPARK-12520) Python API dataframe join returns wrong results on outer join

2015-12-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071695#comment-15071695 ] Xiao Li edited comment on SPARK-12520 at 12/25/15 6:39 PM: --- The problem still

[jira] [Commented] (SPARK-12520) Python API dataframe join returns wrong results on outer join

2015-12-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071695#comment-15071695 ] Xiao Li commented on SPARK-12520: - The problem still exists in 1.4 and 1.5, I think. I did not make a

[jira] [Commented] (SPARK-12520) Python API dataframe join returns wrong results on outer join

2015-12-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071693#comment-15071693 ] Xiao Li commented on SPARK-12520: - You can try it using 1.6. Starting from 1.6, you can specify the join

[jira] [Commented] (SPARK-12521) DataFrame Partitions in java does not work

2015-12-25 Thread Xiu (Joe) Guo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071703#comment-15071703 ] Xiu (Joe) Guo commented on SPARK-12521: --- Thanks [~hvanhovell] to clarifying this up. Maybe it is a

[jira] [Commented] (SPARK-12520) Python API dataframe join returns wrong results on outer join

2015-12-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071688#comment-15071688 ] Xiao Li commented on SPARK-12520: - I can take a look at it. > Python API dataframe join returns wrong

[jira] [Updated] (SPARK-12520) Python API dataframe join returns wrong results on outer join

2015-12-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-12520: Component/s: SQL > Python API dataframe join returns wrong results on outer join >

[jira] [Commented] (SPARK-12520) Python API dataframe join returns wrong results on outer join

2015-12-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071694#comment-15071694 ] Xiao Li commented on SPARK-12520: - I will submit a PR and update the description and test case for Python

[jira] [Commented] (SPARK-12071) Programming guide should explain NULL in JVM translate to NA in R

2015-12-25 Thread Jagadesh Adireddi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071702#comment-15071702 ] Jagadesh Adireddi commented on SPARK-12071: --- If no one is working,I can work on this issue >

[jira] [Commented] (SPARK-12477) [SQL] Tungsten projection fails for null values in array fields

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071754#comment-15071754 ] Apache Spark commented on SPARK-12477: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Updated] (SPARK-12396) Once driver client registered successfully,it still retry to connected to master.

2015-12-25 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-12396: - Affects Version/s: (was: 1.5.2) 1.6.0 > Once driver client registered

[jira] [Issue Comment Deleted] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-25 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-12527: --- Comment: was deleted (was: https://github.com/apache/spark/pull/10482) > Add private val after @transient

[jira] [Created] (SPARK-12526) `ifelse`, `when`, `otherwise` unable to take Column as value

2015-12-25 Thread Sen Fang (JIRA)
Sen Fang created SPARK-12526: Summary: `ifelse`, `when`, `otherwise` unable to take Column as value Key: SPARK-12526 URL: https://issues.apache.org/jira/browse/SPARK-12526 Project: Spark Issue

[jira] [Assigned] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12527: Assignee: Apache Spark > Add private val after @transient for kinesis-asl module >

[jira] [Commented] (SPARK-7159) Support multiclass logistic regression in spark.ml

2015-12-25 Thread George Dittmar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071821#comment-15071821 ] George Dittmar commented on SPARK-7159: --- i'd like to take a stab at this if no one has made progress

[jira] [Created] (SPARK-12525) Fix compiler warnings in Kinesis ASL module due to @transient annotations

2015-12-25 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-12525: -- Summary: Fix compiler warnings in Kinesis ASL module due to @transient annotations Key: SPARK-12525 URL: https://issues.apache.org/jira/browse/SPARK-12525 Project: Spark

[jira] [Resolved] (SPARK-12396) Once driver client registered successfully,it still retry to connected to master.

2015-12-25 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12396. -- Resolution: Fixed Fix Version/s: 2.0.0 > Once driver client registered successfully,it

[jira] [Updated] (SPARK-12396) Once driver client registered successfully,it still retry to connected to master.

2015-12-25 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-12396: - Assignee: echo > Once driver client registered successfully,it still retry to connected to >

[jira] [Assigned] (SPARK-12526) `ifelse`, `when`, `otherwise` unable to take Column as value

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12526: Assignee: Apache Spark > `ifelse`, `when`, `otherwise` unable to take Column as value >

[jira] [Commented] (SPARK-12525) Fix compiler warnings in Kinesis ASL module due to @transient annotations

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071760#comment-15071760 ] Apache Spark commented on SPARK-12525: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12520) Python API dataframe join returns wrong results on outer join

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12520: Assignee: (was: Apache Spark) > Python API dataframe join returns wrong results on

[jira] [Assigned] (SPARK-12520) Python API dataframe join returns wrong results on outer join

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12520: Assignee: Apache Spark > Python API dataframe join returns wrong results on outer join >

[jira] [Created] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-25 Thread Ted Yu (JIRA)
Ted Yu created SPARK-12527: -- Summary: Add private val after @transient for kinesis-asl module Key: SPARK-12527 URL: https://issues.apache.org/jira/browse/SPARK-12527 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12520) Python API dataframe join returns wrong results on outer join

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071709#comment-15071709 ] Apache Spark commented on SPARK-12520: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12525) Fix compiler warnings in Kinesis ASL module due to @transient annotations

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12525: Assignee: Apache Spark (was: Josh Rosen) > Fix compiler warnings in Kinesis ASL module

[jira] [Commented] (SPARK-12469) Consistent Accumulators for Spark

2015-12-25 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071799#comment-15071799 ] Nan Zhu commented on SPARK-12469: - Just to bring the previous discussions about the topic here,

[jira] [Comment Edited] (SPARK-12469) Consistent Accumulators for Spark

2015-12-25 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071799#comment-15071799 ] Nan Zhu edited comment on SPARK-12469 at 12/26/15 2:44 AM: --- Just to bring the

[jira] [Assigned] (SPARK-12526) `ifelse`, `when`, `otherwise` unable to take Column as value

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12526: Assignee: Apache Spark > `ifelse`, `when`, `otherwise` unable to take Column as value >

[jira] [Assigned] (SPARK-12526) `ifelse`, `when`, `otherwise` unable to take Column as value

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12526: Assignee: (was: Apache Spark) > `ifelse`, `when`, `otherwise` unable to take Column

[jira] [Assigned] (SPARK-12526) `ifelse`, `when`, `otherwise` unable to take Column as value

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12526: Assignee: (was: Apache Spark) > `ifelse`, `when`, `otherwise` unable to take Column

[jira] [Commented] (SPARK-12526) `ifelse`, `when`, `otherwise` unable to take Column as value

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071816#comment-15071816 ] Apache Spark commented on SPARK-12526: -- User 'saurfang' has created a pull request for this issue:

[jira] [Updated] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-25 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-12527: --- Description: In SBT build using Scala 2.11, the following warnings were reported which resulted in build

[jira] [Commented] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071819#comment-15071819 ] Apache Spark commented on SPARK-12527: -- User 'ted-yu' has created a pull request for this issue:

[jira] [Commented] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-25 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071818#comment-15071818 ] Ted Yu commented on SPARK-12527: https://github.com/apache/spark/pull/10482 > Add private val after