[jira] [Assigned] (SPARK-1503) Implement Nesterov's accelerated first-order method

[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-1503: Assignee: Xiangrui Meng Implement Nesterov's accelerated first-order method

[jira] [Commented] (SPARK-1503) Implement Nesterov's accelerated first-order method

[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166465#comment-14166465 ] Xiangrui Meng commented on SPARK-1503: -- [~staple] Thanks for picking up this JIRA!

[jira] [Updated] (SPARK-1503) Implement Nesterov's accelerated first-order method

[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1503: - Assignee: Aaron Staple (was: Xiangrui Meng) Implement Nesterov's accelerated first-order method

[jira] [Created] (SPARK-3897) Scala style: format example code

sjk created SPARK-3897: -- Summary: Scala style: format example code Key: SPARK-3897 URL: https://issues.apache.org/jira/browse/SPARK-3897 Project: Spark Issue Type: Sub-task Reporter: sjk

[jira] [Updated] (SPARK-3897) Scala style: format example code

[ https://issues.apache.org/jira/browse/SPARK-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sjk updated SPARK-3897: --- https://github.com/apache/spark/pull/2754 Scala style: format example code

[jira] [Updated] (SPARK-3897) Scala style: format example code

[ https://issues.apache.org/jira/browse/SPARK-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sjk updated SPARK-3897: --- Description: https://github.com/apache/spark/pull/2754 Scala style: format example code

[jira] [Resolved] (SPARK-3421) StructField.toString should quote the name field to allow arbitrary character as struct field name

[ https://issues.apache.org/jira/browse/SPARK-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-3421. --- Resolution: Fixed StructField.toString should quote the name field to allow arbitrary character as

[jira] [Resolved] (SPARK-2805) Update akka to version 2.3.4

[ https://issues.apache.org/jira/browse/SPARK-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2805. Resolution: Fixed We've merged again with some modifications and we'll see if it works well

[jira] [Commented] (SPARK-2019) Spark workers die/disappear when job fails for nearly any reason

[ https://issues.apache.org/jira/browse/SPARK-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166555#comment-14166555 ] Denis Serduik commented on SPARK-2019: -- I have noticed the same problem with workers

[jira] [Comment Edited] (SPARK-2019) Spark workers die/disappear when job fails for nearly any reason

[ https://issues.apache.org/jira/browse/SPARK-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166555#comment-14166555 ] Denis Serduik edited comment on SPARK-2019 at 10/10/14 8:39 AM:

[jira] [Comment Edited] (SPARK-2019) Spark workers die/disappear when job fails for nearly any reason

[ https://issues.apache.org/jira/browse/SPARK-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166555#comment-14166555 ] Denis Serduik edited comment on SPARK-2019 at 10/10/14 8:40 AM:

[jira] [Commented] (SPARK-2348) In Windows having a enviorinment variable named 'classpath' gives error

[ https://issues.apache.org/jira/browse/SPARK-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166559#comment-14166559 ] AaronLin commented on SPARK-2348: - why the issue hasn't been solved? Anyone can help? In

[jira] [Resolved] (SPARK-3889) JVM dies with SIGBUS, resulting in ConnectionManager failed ACK

[ https://issues.apache.org/jira/browse/SPARK-3889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3889. Resolution: Fixed Fix Version/s: 1.2.0 JVM dies with SIGBUS, resulting in ConnectionManager

[jira] [Commented] (SPARK-2348) In Windows having a enviorinment variable named 'classpath' gives error

[ https://issues.apache.org/jira/browse/SPARK-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166566#comment-14166566 ] AaronLin commented on SPARK-2348: - I encounter this issue in sbt building spark1.1.0

[jira] [Created] (SPARK-3898) History Web UI display incorrectly.

zzc created SPARK-3898: -- Summary: History Web UI display incorrectly. Key: SPARK-3898 URL: https://issues.apache.org/jira/browse/SPARK-3898 Project: Spark Issue Type: Bug Components: Web UI

[jira] [Updated] (SPARK-3898) History Web UI display incorrectly.

[ https://issues.apache.org/jira/browse/SPARK-3898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zzc updated SPARK-3898: --- Fix Version/s: 1.2.0 History Web UI display incorrectly. --- Key:

[jira] [Updated] (SPARK-3898) History Web UI display incorrectly.

[ https://issues.apache.org/jira/browse/SPARK-3898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zzc updated SPARK-3898: --- Fix Version/s: 1.1.1 History Web UI display incorrectly. --- Key:

[jira] [Updated] (SPARK-3826) enable hive-thriftserver support hive-0.13.1

[ https://issues.apache.org/jira/browse/SPARK-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-3826: --- Affects Version/s: (was: 1.1.1) 1.1.0 enable hive-thriftserver support

[jira] [Created] (SPARK-3899) wrong links in streaming doc

wangfei created SPARK-3899: -- Summary: wrong links in streaming doc Key: SPARK-3899 URL: https://issues.apache.org/jira/browse/SPARK-3899 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-2429) Hierarchical Implementation of KMeans

[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166593#comment-14166593 ] Yu Ishikawa commented on SPARK-2429: Hi [~rnowling], Thank you for your comments and

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166705#comment-14166705 ] Oleg Zhurakousky commented on SPARK-3561: - Patrick, I think there is

[jira] [Comment Edited] (SPARK-3561) Allow for pluggable execution contexts in Spark

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166705#comment-14166705 ] Oleg Zhurakousky edited comment on SPARK-3561 at 10/10/14 12:10 PM:

[jira] [Commented] (SPARK-3892) Map type should have typeName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166724#comment-14166724 ] Venkata Ramana G commented on SPARK-3892: - Can you explain in detail? Map type

[jira] [Commented] (SPARK-3892) Map type should have typeName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166754#comment-14166754 ] Adrian Wang commented on SPARK-3892: Of course. We are using `.typeName` method to

[jira] [Commented] (SPARK-3892) Map type should have typeName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166812#comment-14166812 ] Cheng Lian commented on SPARK-3892: --- Actually {{MapType.simpleName}} can be simply

[jira] [Commented] (SPARK-3892) Map type should have typeName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166822#comment-14166822 ] Cheng Lian commented on SPARK-3892: --- [~adrian-wang] You're right, it's a typo. So would

[jira] [Updated] (SPARK-3892) Map type should have typeName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-3892: --- Priority: Minor (was: Major) Map type should have typeName -

[jira] [Updated] (SPARK-3892) Map type should have typeName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-3892: --- Issue Type: Improvement (was: Bug) Map type should have typeName -

[jira] [Updated] (SPARK-3892) Map type do not need simpleName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-3892: --- Summary: Map type do not need simpleName (was: Map type should have typeName) Map type do not need

[jira] [Commented] (SPARK-3876) Doing a RDD map/reduce within a DStream map fails with a high enough input rate

[ https://issues.apache.org/jira/browse/SPARK-3876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166832#comment-14166832 ] Saisai Shao commented on SPARK-3876: Hi [~afilip], is there any specific purpose you

[jira] [Commented] (SPARK-3880) HBase as data source to SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-3880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166838#comment-14166838 ] Ravindra Pesala commented on SPARK-3880: There is already some work going in the

[jira] [Commented] (SPARK-3892) Map type do not need simpleName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166840#comment-14166840 ] Adrian Wang commented on SPARK-3892: Yeah. Actually the original method is called

[jira] [Commented] (SPARK-3892) Map type do not need simpleName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166848#comment-14166848 ] Cheng Lian commented on SPARK-3892: --- Ah, while working on the {{DataType}} JSON ser/de

[jira] [Commented] (SPARK-3892) Map type do not need simpleName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166856#comment-14166856 ] Adrian Wang commented on SPARK-3892: Thanks for the explain! I have create PR #2747 to

[jira] [Commented] (SPARK-3892) Map type do not need simpleName

[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166886#comment-14166886 ] Cheng Lian commented on SPARK-3892: --- Please see my comments in the PR :) Map type do

[jira] [Commented] (SPARK-3880) HBase as data source to SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-3880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167006#comment-14167006 ] Yan commented on SPARK-3880: The new context is intended to be very light-weighted. We noticed

[jira] [Updated] (SPARK-3900) ApplicationMaster's shutdown hook fails and IllegalStateException is thrown.

[ https://issues.apache.org/jira/browse/SPARK-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3900: -- Summary: ApplicationMaster's shutdown hook fails and IllegalStateException is thrown. (was:

[jira] [Created] (SPARK-3900) ApplicationMaster's shutdown hook fails to cleanup staging directory.

Kousuke Saruta created SPARK-3900: - Summary: ApplicationMaster's shutdown hook fails to cleanup staging directory. Key: SPARK-3900 URL: https://issues.apache.org/jira/browse/SPARK-3900 Project: Spark

[jira] [Commented] (SPARK-3795) Add scheduler hooks/heuristics for adding and removing executors

[ https://issues.apache.org/jira/browse/SPARK-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167131#comment-14167131 ] Nan Zhu commented on SPARK-3795: this is for YARN or standalone? Add scheduler

[jira] [Resolved] (SPARK-3845) SQLContext(...) should inherit configurations from SparkContext

[ https://issues.apache.org/jira/browse/SPARK-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianshi Huang resolved SPARK-3845. -- Resolution: Fixed Fix Version/s: 1.2.0 SQLContext(...) should inherit configurations

[jira] [Commented] (SPARK-2962) Suboptimal scheduling in spark

[ https://issues.apache.org/jira/browse/SPARK-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167144#comment-14167144 ] Nan Zhu commented on SPARK-2962: Hi, [~mrid...@yahoo-inc.com] I think this has been fixed

[jira] [Commented] (SPARK-3434) Distributed block matrix

[ https://issues.apache.org/jira/browse/SPARK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167152#comment-14167152 ] Burak Yavuz commented on SPARK-3434: [~ConcreteVitamin], any updates? Anything I can

[jira] [Commented] (SPARK-3823) Spark Hive SQL readColumn is not reset each time for a new query

[ https://issues.apache.org/jira/browse/SPARK-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167239#comment-14167239 ] Ravindra Pesala commented on SPARK-3823: It seems this issue is duplicate of

[jira] [Commented] (SPARK-3889) JVM dies with SIGBUS, resulting in ConnectionManager failed ACK

[ https://issues.apache.org/jira/browse/SPARK-3889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167303#comment-14167303 ] Mridul Muralidharan commented on SPARK-3889: The status says fixed - what was

[jira] [Updated] (SPARK-3568) Add metrics for ranking algorithms

[ https://issues.apache.org/jira/browse/SPARK-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-3568: -- Description: Include common metrics for ranking algorithms (http://www-nlp.stanford.edu/IR-book/),

[jira] [Commented] (SPARK-3795) Add scheduler hooks/heuristics for adding and removing executors

[ https://issues.apache.org/jira/browse/SPARK-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167457#comment-14167457 ] Andrew Or commented on SPARK-3795: -- It's agnostic to the cluster manager, but for now we

[jira] [Commented] (SPARK-3434) Distributed block matrix

[ https://issues.apache.org/jira/browse/SPARK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167478#comment-14167478 ] Shivaram Venkataraman commented on SPARK-3434: -- ~brkyvz -- We are just adding

[jira] [Comment Edited] (SPARK-3434) Distributed block matrix

[ https://issues.apache.org/jira/browse/SPARK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167478#comment-14167478 ] Shivaram Venkataraman edited comment on SPARK-3434 at 10/10/14 8:45 PM:

[jira] [Resolved] (SPARK-3886) Choose the batch size of serializer based on size of object

[ https://issues.apache.org/jira/browse/SPARK-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3886. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2740

[jira] [Updated] (SPARK-3886) Choose the batch size of serializer based on size of object

[ https://issues.apache.org/jira/browse/SPARK-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3886: -- Affects Version/s: 1.0.2 1.1.0 Choose the batch size of serializer based on

[jira] [Created] (SPARK-3901) Add SocketSink capability for Spark metrics

Sreepathi Prasanna created SPARK-3901: - Summary: Add SocketSink capability for Spark metrics Key: SPARK-3901 URL: https://issues.apache.org/jira/browse/SPARK-3901 Project: Spark Issue

[jira] [Commented] (SPARK-3901) Add SocketSink capability for Spark metrics

[ https://issues.apache.org/jira/browse/SPARK-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167541#comment-14167541 ] Sreepathi Prasanna commented on SPARK-3901: --- For this, we need a SocketReporter

[jira] [Commented] (SPARK-3901) Add SocketSink capability for Spark metrics

[ https://issues.apache.org/jira/browse/SPARK-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167550#comment-14167550 ] Sreepathi Prasanna commented on SPARK-3901: --- I have the patch ready, but it will

[jira] [Created] (SPARK-3902) Stabilize AsyncRDDActions and expose its methods in Java API

Josh Rosen created SPARK-3902: - Summary: Stabilize AsyncRDDActions and expose its methods in Java API Key: SPARK-3902 URL: https://issues.apache.org/jira/browse/SPARK-3902 Project: Spark Issue

[jira] [Commented] (SPARK-3626) Replace AsyncRDDActions with a more general async. API

[ https://issues.apache.org/jira/browse/SPARK-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167571#comment-14167571 ] Josh Rosen commented on SPARK-3626: --- I've opened SPARK-3902 to discuss stabilizing our

[jira] [Commented] (SPARK-3889) JVM dies with SIGBUS, resulting in ConnectionManager failed ACK

[ https://issues.apache.org/jira/browse/SPARK-3889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167647#comment-14167647 ] Aaron Davidson commented on SPARK-3889: --- Sorry, it was not linked:

[jira] [Updated] (SPARK-3855) Binding Exception when running PythonUDFs

[ https://issues.apache.org/jira/browse/SPARK-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3855: --- Component/s: PySpark Binding Exception when running PythonUDFs

[jira] [Updated] (SPARK-3855) Binding Exception when running PythonUDFs

[ https://issues.apache.org/jira/browse/SPARK-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3855: --- Component/s: SQL Binding Exception when running PythonUDFs

[jira] [Commented] (SPARK-1503) Implement Nesterov's accelerated first-order method

[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167868#comment-14167868 ] Aaron Staple commented on SPARK-1503: - [~mengxr] Thanks for the heads up! I’ll

[jira] [Created] (SPARK-3903) Create general data loading method for LabeledPoints

Joseph K. Bradley created SPARK-3903: Summary: Create general data loading method for LabeledPoints Key: SPARK-3903 URL: https://issues.apache.org/jira/browse/SPARK-3903 Project: Spark

[jira] [Updated] (SPARK-3903) Create general data loading method for LabeledPoints

[ https://issues.apache.org/jira/browse/SPARK-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3903: - Description: Proposal: Provide a more general data loading function for LabeledPoints. *

[jira] [Updated] (SPARK-3903) Create general data loading method for LabeledPoints

[ https://issues.apache.org/jira/browse/SPARK-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3903: - Assignee: Joseph K. Bradley Create general data loading method for LabeledPoints

[jira] [Updated] (SPARK-3898) History Web UI display incorrectly.

[ https://issues.apache.org/jira/browse/SPARK-3898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zzc updated SPARK-3898: --- Fix Version/s: (was: 1.1.1) History Web UI display incorrectly. ---

[jira] [Updated] (SPARK-3586) Support nested directories in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxj updated SPARK-3586: -- Issue Type: Bug (was: Improvement) Support nested directories in Spark Streaming

[jira] [Updated] (SPARK-3867) ./python/run-tests failed when it run with Python 2.6 and unittest2 is not installed

[ https://issues.apache.org/jira/browse/SPARK-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cocoatomo updated SPARK-3867: - Description: ./python/run-tests search a Python 2.6 executable on PATH and use it if available. When

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168018#comment-14168018 ] Davies Liu commented on SPARK-922: -- We did not use json heavily in pyspark, also user have