[jira] [Created] (SPARK-4783) Remove all System.exit calls from sparkcontext

David Semeria created SPARK-4783: Summary: Remove all System.exit calls from sparkcontext Key: SPARK-4783 URL: https://issues.apache.org/jira/browse/SPARK-4783 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4783) Remove all System.exit calls from sparkcontext

[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Semeria updated SPARK-4783: - Description: A common architectural choice for integrating Spark within a larger application is t

[jira] [Commented] (SPARK-2808) update kafka to version 0.8.2

[ https://issues.apache.org/jira/browse/SPARK-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237150#comment-14237150 ] Helena Edelson commented on SPARK-2808: --- I've done the migration, am testing the cha

[jira] [Commented] (SPARK-4181) Create separate options to control the client-mode AM resource allocation request

[ https://issues.apache.org/jira/browse/SPARK-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237163#comment-14237163 ] WangTaoTheTonic commented on SPARK-4181: I couldn't think out one case for extraLi

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237178#comment-14237178 ] Zhang, Liye commented on SPARK-4740: [~pwendell], I have the same concern first, and I

[jira] [Comment Edited] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237178#comment-14237178 ] Zhang, Liye edited comment on SPARK-4740 at 12/7/14 3:32 PM: -

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237179#comment-14237179 ] Zhang, Liye commented on SPARK-4740: Hi [~adav], I kept speculation as default (false)

[jira] [Comment Edited] (SPARK-2808) update kafka to version 0.8.2

[ https://issues.apache.org/jira/browse/SPARK-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237150#comment-14237150 ] Helena Edelson edited comment on SPARK-2808 at 12/7/14 3:35 PM:

[jira] [Comment Edited] (SPARK-2808) update kafka to version 0.8.2

[ https://issues.apache.org/jira/browse/SPARK-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237150#comment-14237150 ] Helena Edelson edited comment on SPARK-2808 at 12/7/14 3:52 PM:

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237233#comment-14237233 ] koert kuipers commented on SPARK-3655: -- [~sandyr] One issue with groupByKeyAndSortVal

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237254#comment-14237254 ] koert kuipers commented on SPARK-3655: -- i also dont like the signature def groupByKe

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode with multiple cores

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237264#comment-14237264 ] Davis Shepherd commented on SPARK-4759: --- It is possible to reproduce the issue with

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237298#comment-14237298 ] Apache Spark commented on SPARK-3655: - User 'koertkuipers' has created a pull request

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237299#comment-14237299 ] koert kuipers commented on SPARK-3655: -- i have a new pullreq that implements just gro

[jira] [Updated] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koert kuipers updated SPARK-3655: - Affects Version/s: 1.2.0 > Support sorting of values in addition to keys (i.e. secondary sort) > -

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode with multiple cores

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or commented on SPARK-4759: -- Hey I came up with a much smaller reproduction for th

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode with multiple cores

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/7/14 11:53 PM: He

[jira] [Updated] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4759: - Summary: Deadlock in complex spark job in local mode (was: Deadlock in complex spark job in local mode wi

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode with multiple cores

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/7/14 11:53 PM: He

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/7/14 11:57 PM: He

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 12:17 AM: He

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 12:20 AM: He

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 12:20 AM: He

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 12:25 AM: He

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 12:36 AM: He

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 12:36 AM: He

[jira] [Commented] (SPARK-4783) Remove all System.exit calls from sparkcontext

[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237341#comment-14237341 ] Patrick Wendell commented on SPARK-4783: For code cleanliness, we should go throug

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 1:29 AM: --- Hey

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 1:30 AM: --- Hey

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 1:33 AM: --- Hey

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237371#comment-14237371 ] Davis Shepherd commented on SPARK-4759: --- This doesn't seem to reproduce the issue fo

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237372#comment-14237372 ] Andrew Or commented on SPARK-4759: -- Found the issue. The task scheduler schedules tasks b

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237374#comment-14237374 ] Andrew Or commented on SPARK-4759: -- [~dgshep] That's strange. I am able to reproduce this

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237376#comment-14237376 ] Davis Shepherd commented on SPARK-4759: --- local[2] on the latest commit of branch 1.1

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237374#comment-14237374 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 2:36 AM: --- [~dg

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237377#comment-14237377 ] Andrew Or commented on SPARK-4759: -- Hm I'll try branch 1.1 again later tonight. There mig

[jira] [Updated] (SPARK-4783) System.exit() calls in SparkContext disrupt applications embedding Spark

[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4783: --- Summary: System.exit() calls in SparkContext disrupt applications embedding Spark (was: Remov

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237383#comment-14237383 ] Davis Shepherd commented on SPARK-4759: --- Ok your version does reproduce the issue ag

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237393#comment-14237393 ] Davis Shepherd commented on SPARK-4759: --- I still cannot reproduce the issue in the s

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237393#comment-14237393 ] Davis Shepherd edited comment on SPARK-4759 at 12/8/14 3:15 AM:

[jira] [Comment Edited] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237299#comment-14237299 ] koert kuipers edited comment on SPARK-3655 at 12/8/14 3:24 AM: -

[jira] [Resolved] (SPARK-4646) Replace Scala.util.Sorting.quickSort with Sorter(TimSort) in Spark

[ https://issues.apache.org/jira/browse/SPARK-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-4646. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3507 [https://github.com/

[jira] [Resolved] (SPARK-4620) Add unpersist in Graph/GraphImpl

[ https://issues.apache.org/jira/browse/SPARK-4620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-4620. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3476 [https://github.com/

[jira] [Created] (SPARK-4784) Model.fittingParamMap should store all Params

Joseph K. Bradley created SPARK-4784: Summary: Model.fittingParamMap should store all Params Key: SPARK-4784 URL: https://issues.apache.org/jira/browse/SPARK-4784 Project: Spark Issue Typ

[jira] [Updated] (SPARK-4784) Model.fittingParamMap should store all Params

[ https://issues.apache.org/jira/browse/SPARK-4784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-4784: - Description: spark.ml's Model class should store all parameters in the fittingParamMap, no

[jira] [Commented] (SPARK-1350) YARN ContainerLaunchContext should use cluster's JAVA_HOME

[ https://issues.apache.org/jira/browse/SPARK-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237525#comment-14237525 ] Aniket Bhatnagar commented on SPARK-1350: - [~sandyr] Using Environment.JAVA_HOME.$

[jira] [Created] (SPARK-4785) When called with arguments referring column fields, PMOD throws NPE

Cheng Lian created SPARK-4785: - Summary: When called with arguments referring column fields, PMOD throws NPE Key: SPARK-4785 URL: https://issues.apache.org/jira/browse/SPARK-4785 Project: Spark

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237550#comment-14237550 ] Andrew Or commented on SPARK-4759: -- Ok yeah you're right, I can't reproduce it from the c

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 7:28 AM: --- Hey

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 7:29 AM: --- Hey

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 7:29 AM: --- Hey

[jira] [Updated] (SPARK-4781) Column values become all NULL after doing ALTER TABLE CHANGE for renaming column names (Parquet external table in HiveContext)

[ https://issues.apache.org/jira/browse/SPARK-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianshi Huang updated SPARK-4781: - Issue Type: Bug (was: Improvement) > Column values become all NULL after doing ALTER TABLE CHANGE