[jira] [Commented] (SPARK-3530) Pipeline and Parameters

[ https://issues.apache.org/jira/browse/SPARK-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138319#comment-14138319 ] Sandy Ryza commented on SPARK-3530: --- bq. Isn't the fit multiple models at once part a

[jira] [Updated] (SPARK-2620) case class cannot be used as key for reduce

[ https://issues.apache.org/jira/browse/SPARK-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2620: - Affects Version/s: 1.1.0 case class cannot be used as key for reduce

[jira] [Commented] (SPARK-3574) Shuffle finish time always reported as -1

[ https://issues.apache.org/jira/browse/SPARK-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138330#comment-14138330 ] Apache Spark commented on SPARK-3574: - User 'sryza' has created a pull request for

[jira] [Commented] (SPARK-3577) Shuffle write time incorrect for sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138346#comment-14138346 ] Kay Ousterhout commented on SPARK-3577: --- I noticed it in the code when I was trying

[jira] [Commented] (SPARK-3577) Shuffle write time incorrect for sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138354#comment-14138354 ] Sandy Ryza commented on SPARK-3577: --- In the old code, the ShuffleWriteMetrics didn't get

[jira] [Commented] (SPARK-3292) Shuffle Tasks run incessantly even though there's no inputs

[ https://issues.apache.org/jira/browse/SPARK-3292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138371#comment-14138371 ] guowei commented on SPARK-3292: --- below is where i changed in Dstream whether commit job or

[jira] [Updated] (SPARK-3577) Add task metric to report spill time

[ https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-3577: -- Description: The ExternalSorter passes its own ShuffleWriteMetrics into ExternalSorter.

[jira] [Commented] (SPARK-3563) Shuffle data not always be cleaned

[ https://issues.apache.org/jira/browse/SPARK-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138376#comment-14138376 ] shenhong commented on SPARK-3563: - I have just trigger the driver a full GC, and the

[jira] [Comment Edited] (SPARK-3563) Shuffle data not always be cleaned

[ https://issues.apache.org/jira/browse/SPARK-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138376#comment-14138376 ] shenhong edited comment on SPARK-3563 at 9/18/14 1:47 AM: -- I have

[jira] [Commented] (SPARK-3563) Shuffle data not always be cleaned

[ https://issues.apache.org/jira/browse/SPARK-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138389#comment-14138389 ] Saisai Shao commented on SPARK-3563: I think it relies on JVM's GC strategy to treat

[jira] [Commented] (SPARK-2593) Add ability to pass an existing Akka ActorSystem into Spark

[ https://issues.apache.org/jira/browse/SPARK-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138390#comment-14138390 ] Matei Zaharia commented on SPARK-2593: -- The reason that we don't want to expose Akka

[jira] [Commented] (SPARK-2593) Add ability to pass an existing Akka ActorSystem into Spark

[ https://issues.apache.org/jira/browse/SPARK-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138402#comment-14138402 ] Matei Zaharia commented on SPARK-2593: -- BTW doing this for the ActorReceiver for

[jira] [Comment Edited] (SPARK-3563) Shuffle data not always be cleaned

[ https://issues.apache.org/jira/browse/SPARK-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137348#comment-14137348 ] shenhong edited comment on SPARK-3563 at 9/18/14 2:16 AM: --

[jira] [Closed] (SPARK-2022) Spark 1.0.0 is failing if mesos.coarse set to true

[ https://issues.apache.org/jira/browse/SPARK-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Farrellee closed SPARK-2022. Resolution: Fixed Spark 1.0.0 is failing if mesos.coarse set to true

[jira] [Closed] (SPARK-3416) Add matrix operations for large data set

[ https://issues.apache.org/jira/browse/SPARK-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa closed SPARK-3416. -- Resolution: Duplicate Add matrix operations for large data set

[jira] [Commented] (SPARK-3416) Add matrix operations for large data set

[ https://issues.apache.org/jira/browse/SPARK-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138445#comment-14138445 ] Yu Ishikawa commented on SPARK-3416: Hi [~srowen], Yes. I'm sorry I forget to close

[jira] [Commented] (SPARK-3534) Avoid running MLlib and Streaming tests when testing SQL PRs

[ https://issues.apache.org/jira/browse/SPARK-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138459#comment-14138459 ] Apache Spark commented on SPARK-3534: - User 'nchammas' has created a pull request for

[jira] [Updated] (SPARK-3580) Add Consistent Method To Get Number of RDD Partitions Across Different Languages

[ https://issues.apache.org/jira/browse/SPARK-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3580: --- Labels: starter (was: ) Add Consistent Method To Get Number of RDD Partitions Across

[jira] [Commented] (SPARK-3580) Add Consistent Method To Get Number of RDD Partitions Across Different Languages

[ https://issues.apache.org/jira/browse/SPARK-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138460#comment-14138460 ] Patrick Wendell commented on SPARK-3580: Yeah I think it's a good idea to add

[jira] [Updated] (SPARK-3579) Jekyll doc generation is different across environments

[ https://issues.apache.org/jira/browse/SPARK-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3579: --- Description: This can result in a lot of false changes when someone alters something with

[jira] [Commented] (SPARK-3578) GraphGenerators.sampleLogNormal sometimes returns too-large result

[ https://issues.apache.org/jira/browse/SPARK-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138480#comment-14138480 ] Patrick Wendell commented on SPARK-3578: @ankurdave, could you tag stuff as GraphX

[jira] [Updated] (SPARK-3578) GraphGenerators.sampleLogNormal sometimes returns too-large result

[ https://issues.apache.org/jira/browse/SPARK-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3578: --- Component/s: GraphX GraphGenerators.sampleLogNormal sometimes returns too-large result

[jira] [Commented] (SPARK-3579) Jekyll doc generation is different across environments

[ https://issues.apache.org/jira/browse/SPARK-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138505#comment-14138505 ] Apache Spark commented on SPARK-3579: - User 'pwendell' has created a pull request for

[jira] [Updated] (SPARK-3566) .gitignore and .rat-excludes should consider Windows cmd file and Emacs' backup files

[ https://issues.apache.org/jira/browse/SPARK-3566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3566: -- Summary: .gitignore and .rat-excludes should consider Windows cmd file and Emacs' backup files

[jira] [Commented] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138547#comment-14138547 ] Saisai Shao commented on SPARK-2926: Looking forward to your feedback :). Add

[jira] [Resolved] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-. Resolution: Fixed This was documented in the release upgrade notes, so I think we're all

[jira] [Updated] (SPARK-3582) Spark SQL having issue with existing Hive UDFs which take Map as a parameter

[ https://issues.apache.org/jira/browse/SPARK-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Santhosh updated SPARK-3582: Summary: Spark SQL having issue with existing Hive UDFs which take Map as a parameter

[jira] [Created] (SPARK-3581) RDD API(distinct/subtract) does not work for RDD of Dictionaries

Shawn Guo created SPARK-3581: Summary: RDD API(distinct/subtract) does not work for RDD of Dictionaries Key: SPARK-3581 URL: https://issues.apache.org/jira/browse/SPARK-3581 Project: Spark

[jira] [Created] (SPARK-3582) Spark SQL hving issue with existing Hive UDFs which take Map as a parameter

Saurabh Santhosh created SPARK-3582: --- Summary: Spark SQL hving issue with existing Hive UDFs which take Map as a parameter Key: SPARK-3582 URL: https://issues.apache.org/jira/browse/SPARK-3582

[jira] [Updated] (SPARK-3581) RDD API(distinct/subtract) does not work for RDD of Dictionaries

[ https://issues.apache.org/jira/browse/SPARK-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Guo updated SPARK-3581: - Description: Construct a RDD of dictionaries(dictRDD), try to use the RDD API, RDD.distinct() or

[jira] [Updated] (SPARK-3581) RDD API(distinct/subtract) does not work for RDD of Dictionaries

[ https://issues.apache.org/jira/browse/SPARK-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Guo updated SPARK-3581: - Description: Construct a RDD of dictionaries(dictRDD), try to use the RDD API, RDD.distinct() or

[jira] [Commented] (SPARK-3582) Spark SQL having issue with existing Hive UDFs which take Map as a parameter

[ https://issues.apache.org/jira/browse/SPARK-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138559#comment-14138559 ] Saurabh Santhosh commented on SPARK-3582: - When i changed the parameter to Object

<    1   2