[jira] [Commented] (SPARK-4595) Spark MetricsServlet is not worked because of initialization ordering

[ https://issues.apache.org/jira/browse/SPARK-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224177#comment-14224177 ] Saisai Shao commented on SPARK-4595: Hi Josh, I think this bug may probably be introd

[jira] [Commented] (SPARK-4395) Running a Spark SQL SELECT command from PySpark causes a hang for ~ 1 hour

[ https://issues.apache.org/jira/browse/SPARK-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224183#comment-14224183 ] Cheng Lian commented on SPARK-4395: --- Couldn't reproduce the long pause locally, and unfo

[jira] [Commented] (SPARK-4592) "Worker registration failed: Duplicate worker ID" error during Master failover

[ https://issues.apache.org/jira/browse/SPARK-4592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224184#comment-14224184 ] Apache Spark commented on SPARK-4592: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224185#comment-14224185 ] Sandy Ryza commented on SPARK-4584: --- I took a look at the jobs Nishkam ran before and af

[jira] [Updated] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4584: -- Target Version/s: 1.2.0 > 2x Performance regression for Spark-on-YARN >

[jira] [Commented] (SPARK-3588) Gaussian Mixture Model clustering

[ https://issues.apache.org/jira/browse/SPARK-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224188#comment-14224188 ] Xiangrui Meng commented on SPARK-3588: -- Since [~tgaloppo] already submitted a PR, we

[jira] [Commented] (SPARK-4592) "Worker registration failed: Duplicate worker ID" error during Master failover

[ https://issues.apache.org/jira/browse/SPARK-4592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224194#comment-14224194 ] Andrew Or commented on SPARK-4592: -- I submitted a fix at https://github.com/apache/spark/

[jira] [Commented] (SPARK-4594) Improvement the broadcast for HiveConf

[ https://issues.apache.org/jira/browse/SPARK-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224207#comment-14224207 ] Apache Spark commented on SPARK-4594: - User 'Leolh' has created a pull request for thi

[jira] [Created] (SPARK-4597) Use proper exception and reset variable

Liang-Chi Hsieh created SPARK-4597: -- Summary: Use proper exception and reset variable Key: SPARK-4597 URL: https://issues.apache.org/jira/browse/SPARK-4597 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4597) Use proper exception and reset variable

[ https://issues.apache.org/jira/browse/SPARK-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224215#comment-14224215 ] Apache Spark commented on SPARK-4597: - User 'viirya' has created a pull request for th

[jira] [Created] (SPARK-4598) java.lang.OutOfMemoryError occurs when opening stage page of an application has 100000 tasks,

meiyoula created SPARK-4598: --- Summary: java.lang.OutOfMemoryError occurs when opening stage page of an application has 10 tasks, Key: SPARK-4598 URL: https://issues.apache.org/jira/browse/SPARK-4598 Pr

[jira] [Commented] (SPARK-4585) Spark dynamic scaling executors use upper limit value as default.

[ https://issues.apache.org/jira/browse/SPARK-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224244#comment-14224244 ] Sean Owen commented on SPARK-4585: -- Given the discussion in SPARK-3174, it seems like thi

[jira] [Created] (SPARK-4599) add hive profile to root pom

Adrian Wang created SPARK-4599: -- Summary: add hive profile to root pom Key: SPARK-4599 URL: https://issues.apache.org/jira/browse/SPARK-4599 Project: Spark Issue Type: Improvement Comp

[jira] [Updated] (SPARK-4585) Spark dynamic scaling executors use upper limit value as default.

[ https://issues.apache.org/jira/browse/SPARK-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-4585: -- Issue Type: Improvement (was: Bug) > Spark dynamic scaling executors use upper limit value as default.

[jira] [Commented] (SPARK-4599) add hive profile to root pom

[ https://issues.apache.org/jira/browse/SPARK-4599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224256#comment-14224256 ] Apache Spark commented on SPARK-4599: - User 'adrian-wang' has created a pull request f

[jira] [Commented] (SPARK-4585) Spark dynamic scaling executors use upper limit value as default.

[ https://issues.apache.org/jira/browse/SPARK-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224257#comment-14224257 ] Sandy Ryza commented on SPARK-4585: --- I was discussing this with [~brocknoland]. The iss

[jira] [Resolved] (SPARK-962) debian package contains old version of executable scripts

[ https://issues.apache.org/jira/browse/SPARK-962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-962. - Resolution: Fixed This appears fixed. The referenced PR was apparently merged into 0.9, and the outdated s

[jira] [Assigned] (SPARK-4509) Revert EC2 tag-based cluster membership patch in branch-1.2

[ https://issues.apache.org/jira/browse/SPARK-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-4509: Assignee: Xiangrui Meng > Revert EC2 tag-based cluster membership patch in branch-1.2 > ---

[jira] [Commented] (SPARK-4594) Improvement the broadcast for HiveConf

[ https://issues.apache.org/jira/browse/SPARK-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224284#comment-14224284 ] Apache Spark commented on SPARK-4594: - User 'Leolh' has created a pull request for thi

[jira] [Commented] (SPARK-3588) Gaussian Mixture Model clustering

[ https://issues.apache.org/jira/browse/SPARK-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224287#comment-14224287 ] Travis Galoppo commented on SPARK-3588: --- Sorry about the duplicate effort; I did a s

[jira] [Assigned] (SPARK-4592) "Worker registration failed: Duplicate worker ID" error during Master failover

[ https://issues.apache.org/jira/browse/SPARK-4592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-4592: Assignee: Andrew Or (was: Josh Rosen) > "Worker registration failed: Duplicate worker ID" error du

[jira] [Resolved] (SPARK-910) hadoopFile creates RecordReader key and value at the wrong scope

[ https://issues.apache.org/jira/browse/SPARK-910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-910. - Resolution: Not a Problem Given the PR discussion, it looks like this was resolved as NotAProblem. Either

[jira] [Commented] (SPARK-4509) Revert EC2 tag-based cluster membership patch in branch-1.2

[ https://issues.apache.org/jira/browse/SPARK-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224302#comment-14224302 ] Apache Spark commented on SPARK-4509: - User 'mengxr' has created a pull request for th

[jira] [Resolved] (SPARK-981) Seemingly spurious "Duplicate worker ID" error messages

[ https://issues.apache.org/jira/browse/SPARK-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-981. - Resolution: Duplicate It sounds like it may be the same issue reported, and being worked on, in https://is

[jira] [Commented] (SPARK-1148) Suggestions for exception handling (avoid potential bugs)

[ https://issues.apache.org/jira/browse/SPARK-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224305#comment-14224305 ] Sean Owen commented on SPARK-1148: -- [~d.yuan] Several problems like this have been fixed

[jira] [Updated] (SPARK-4596) Refactorize Normalizer to make code cleaner

[ https://issues.apache.org/jira/browse/SPARK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4596: - Assignee: DB Tsai > Refactorize Normalizer to make code cleaner >

[jira] [Resolved] (SPARK-4596) Refactorize Normalizer to make code cleaner

[ https://issues.apache.org/jira/browse/SPARK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4596. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3446 [https://githu

[jira] [Updated] (SPARK-4530) GradientDescent get a wrong gradient value according to the gradient formula, which is caused by the miniBatchSize parameter.

[ https://issues.apache.org/jira/browse/SPARK-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4530: - Fix Version/s: 1.2.0 > GradientDescent get a wrong gradient value according to the gradient formul

[jira] [Commented] (SPARK-4530) GradientDescent get a wrong gradient value according to the gradient formula, which is caused by the miniBatchSize parameter.

[ https://issues.apache.org/jira/browse/SPARK-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224312#comment-14224312 ] Xiangrui Meng commented on SPARK-4530: -- PR: https://github.com/apache/spark/pull/3399

[jira] [Commented] (SPARK-4526) Gradient should be added batch computing interface

[ https://issues.apache.org/jira/browse/SPARK-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224314#comment-14224314 ] Xiangrui Meng commented on SPARK-4526: -- PR #3399 was created for SPARK-4530. > Gradi

[jira] [Updated] (SPARK-1182) Sort the configuration parameters in configuration.md

[ https://issues.apache.org/jira/browse/SPARK-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-1182: - Fix Version/s: (was: 1.0.0) The current PR seems to be https://github.com/apache/spark/pull/2312 but i

[jira] [Commented] (SPARK-1016) When running examples jar (compiled with maven) logs don't initialize properly

[ https://issues.apache.org/jira/browse/SPARK-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224336#comment-14224336 ] Sean Owen commented on SPARK-1016: -- Is this resolved now? examples should be logging via

[jira] [Commented] (SPARK-4530) GradientDescent get a wrong gradient value according to the gradient formula, which is caused by the miniBatchSize parameter.

[ https://issues.apache.org/jira/browse/SPARK-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224378#comment-14224378 ] Apache Spark commented on SPARK-4530: - User 'witgo' has created a pull request for thi

[jira] [Commented] (SPARK-1450) Specify the default zone in the EC2 script help

[ https://issues.apache.org/jira/browse/SPARK-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224384#comment-14224384 ] Apache Spark commented on SPARK-1450: - User 'srowen' has created a pull request for th

[jira] [Commented] (SPARK-1301) Add UI elements to collapse "Aggregated Metrics by Executor" pane on stage page

[ https://issues.apache.org/jira/browse/SPARK-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224387#comment-14224387 ] Sean Owen commented on SPARK-1301: -- Is this still relevant now that this info is on a sep

[jira] [Resolved] (SPARK-2002) Race condition in accessing cache locations in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-2002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2002. -- Resolution: Duplicate This appears to be identical to SPARK-4454, and that one has an open PR. > Race c

[jira] [Resolved] (SPARK-2223) Building and running tests with maven is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2223. -- Resolution: Duplicate Can I suggest this be folded into SPARK-3431? there's no action here, and the act

[jira] [Created] (SPARK-4600) org.apache.spark.graphx.VertexRDD.diff does not work

Teppei Tosa created SPARK-4600: -- Summary: org.apache.spark.graphx.VertexRDD.diff does not work Key: SPARK-4600 URL: https://issues.apache.org/jira/browse/SPARK-4600 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-2192) Examples Data Not in Binary Distribution

[ https://issues.apache.org/jira/browse/SPARK-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224436#comment-14224436 ] Sean Owen commented on SPARK-2192: -- Data files are now consolidated under "data/", and th

[jira] [Resolved] (SPARK-2556) Multiple SparkContexts can coexist in one process

[ https://issues.apache.org/jira/browse/SPARK-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2556. -- Resolution: Duplicate > Multiple SparkContexts can coexist in one process >

[jira] [Resolved] (SPARK-1623) SPARK-1623. Broadcast cleaner should use getCanonicalPath when deleting files by name

[ https://issues.apache.org/jira/browse/SPARK-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1623. -- Resolution: Fixed Yes, the original PR was clearly superseded by https://github.com/apache/spark/pull/7

[jira] [Resolved] (SPARK-2404) spark-submit and spark-class may overwrite the already defined SPARK_HOME

[ https://issues.apache.org/jira/browse/SPARK-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2404. -- Resolution: Won't Fix Fix Version/s: (was: 1.0.1) According to the PR discussion, this appear

[jira] [Resolved] (SPARK-4535) Fix the error in comments

[ https://issues.apache.org/jira/browse/SPARK-4535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4535. -- Resolution: Fixed Fix Version/s: 1.2.0 > Fix the error in comments >

[jira] [Resolved] (SPARK-2290) Do not send SPARK_HOME from driver to executors

[ https://issues.apache.org/jira/browse/SPARK-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2290. -- Resolution: Duplicate According to PR discussion, this duplicates SPARK-2454, which was resolved in htt

[jira] [Resolved] (SPARK-3009) ApplicationInfo doesn't get initialised after deserialisation during recovery

[ https://issues.apache.org/jira/browse/SPARK-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3009. -- Resolution: Fixed Looks like https://github.com/apache/spark/pull/1947 was merged and resolved this. >

[jira] [Commented] (SPARK-4036) Add Conditional Random Fields (CRF) algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224467#comment-14224467 ] Kai Sasaki commented on SPARK-4036: --- Hi, I want to work on this ticket. I'll write a bas

[jira] [Commented] (SPARK-3171) Don't print meaningless information of SelectionKey

[ https://issues.apache.org/jira/browse/SPARK-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224469#comment-14224469 ] Sean Owen commented on SPARK-3171: -- Looks like this was ready to commit but the PR was cl

[jira] [Resolved] (SPARK-2419) Misc updates to streaming programming guide

[ https://issues.apache.org/jira/browse/SPARK-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2419. -- Resolution: Fixed Looks like both of the PRs were merged, so this is resolved. > Misc updates to stream

[jira] [Resolved] (SPARK-2027) spark-ec2 puts Hadoop's log4j ahead of Spark's in classpath

[ https://issues.apache.org/jira/browse/SPARK-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2027. -- Resolution: Duplicate > spark-ec2 puts Hadoop's log4j ahead of Spark's in classpath > --

[jira] [Resolved] (SPARK-2007) Spark on YARN picks up hadoop log4j.properties even if SPARK_LOG4J_CONF is set

[ https://issues.apache.org/jira/browse/SPARK-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2007. -- Resolution: Fixed Looks like this is resolved per the method Marcelo alludes to in the PR. That seems t

[jira] [Commented] (SPARK-2132) Color GC time red when over a percentage of task time

[ https://issues.apache.org/jira/browse/SPARK-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224490#comment-14224490 ] Sean Owen commented on SPARK-2132: -- This might be best as part of the larger https://iss

[jira] [Created] (SPARK-4601) Call site of jobs generated by streaming incorrect in Spark UI

Tathagata Das created SPARK-4601: Summary: Call site of jobs generated by streaming incorrect in Spark UI Key: SPARK-4601 URL: https://issues.apache.org/jira/browse/SPARK-4601 Project: Spark

[jira] [Updated] (SPARK-4601) Call site of jobs generated by streaming incorrect in Spark UI

[ https://issues.apache.org/jira/browse/SPARK-4601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4601: - Description: When running the NetworkWordCount, the description of the word count jobs are set as

[jira] [Commented] (SPARK-4601) Call site of jobs generated by streaming incorrect in Spark UI

[ https://issues.apache.org/jira/browse/SPARK-4601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224523#comment-14224523 ] Apache Spark commented on SPARK-4601: - User 'tdas' has created a pull request for this

[jira] [Commented] (SPARK-4598) java.lang.OutOfMemoryError occurs when opening stage page of an application has 100000 tasks,

[ https://issues.apache.org/jira/browse/SPARK-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224529#comment-14224529 ] Apache Spark commented on SPARK-4598: - User 'XuTingjun' has created a pull request for

[jira] [Commented] (SPARK-2475) Check whether #cores > #receivers in local mode

[ https://issues.apache.org/jira/browse/SPARK-2475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224537#comment-14224537 ] Tathagata Das commented on SPARK-2475: -- SPARK-4381 partially solved for the most comm

[jira] [Updated] (SPARK-4381) User should get warned when set spark.master with local in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4381: - Priority: Major (was: Minor) > User should get warned when set spark.master with local in Spark S

[jira] [Resolved] (SPARK-3024) CLI interface to Driver

[ https://issues.apache.org/jira/browse/SPARK-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3024. -- Resolution: Duplicate If the gist of this is to expose the UI data via JSON, can I suggest this be fold

[jira] [Resolved] (SPARK-4381) User should get warned when set spark.master with local in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4381. -- Resolution: Fixed Fix Version/s: 1.3.0 1.2.0 > User should get warned

[jira] [Updated] (SPARK-4381) User should get warned when set spark.master with local in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4381: - Affects Version/s: (was: 1.2.0) 1.1.1 1.0.2 > Us

[jira] [Commented] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

[ https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224539#comment-14224539 ] Tathagata Das commented on SPARK-4133: -- [~joshrosen] Any more comments regarding this

[jira] [Updated] (SPARK-2985) Buffered data in BlockGenerator gets lost when receiver crashes

[ https://issues.apache.org/jira/browse/SPARK-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2985: - Summary: Buffered data in BlockGenerator gets lost when receiver crashes (was: BlockGenerator not

[jira] [Commented] (SPARK-2985) Buffered data in BlockGenerator gets lost when receiver crashes

[ https://issues.apache.org/jira/browse/SPARK-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224555#comment-14224555 ] Tathagata Das commented on SPARK-2985: -- This is by design. If you are using the recei

[jira] [Commented] (SPARK-4314) Exception when textFileStream attempts to read deleted _COPYING_ file

[ https://issues.apache.org/jira/browse/SPARK-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224559#comment-14224559 ] Tathagata Das commented on SPARK-4314: -- Let me try to understand the scenario when th

[jira] [Commented] (SPARK-4462) flume-sink build broken in SBT

[ https://issues.apache.org/jira/browse/SPARK-4462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224564#comment-14224564 ] Tathagata Das commented on SPARK-4462: -- [~marmbrus] Are you still seeing this issue?

[jira] [Commented] (SPARK-4276) Spark streaming requires at least two working thread

[ https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224565#comment-14224565 ] Tathagata Das commented on SPARK-4276: -- This issue is resolved by this JIRA, the user

[jira] [Commented] (SPARK-4276) Spark streaming requires at least two working thread

[ https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224566#comment-14224566 ] Tathagata Das commented on SPARK-4276: -- Since this is not an issue, I am closing this

[jira] [Closed] (SPARK-4276) Spark streaming requires at least two working thread

[ https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das closed SPARK-4276. Resolution: Duplicate > Spark streaming requires at least two working thread > -

[jira] [Commented] (SPARK-2383) With auto.offset.reset, KafkaReceiver potentially deletes Consumer nodes from Zookeeper

[ https://issues.apache.org/jira/browse/SPARK-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224571#comment-14224571 ] Tathagata Das commented on SPARK-2383: -- This issue has been resolved in Spark 1.2.0.

[jira] [Closed] (SPARK-2383) With auto.offset.reset, KafkaReceiver potentially deletes Consumer nodes from Zookeeper

[ https://issues.apache.org/jira/browse/SPARK-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das closed SPARK-2383. Resolution: Duplicate > With auto.offset.reset, KafkaReceiver potentially deletes Consumer nodes fro

[jira] [Commented] (SPARK-2401) AdaBoost.MH, a multi-class multi-label classifier

[ https://issues.apache.org/jira/browse/SPARK-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224575#comment-14224575 ] Sean Owen commented on SPARK-2401: -- This looks like a duplicate of SPARK-1546. At least t

[jira] [Commented] (SPARK-4537) Add 'processing delay' and 'totalDelay' to the metrics reported by the Spark Streaming subsystem

[ https://issues.apache.org/jira/browse/SPARK-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224580#comment-14224580 ] Tathagata Das commented on SPARK-4537: -- Would be cool to solve this issue! > Add 'pr

[jira] [Commented] (SPARK-4196) Streaming + checkpointing + saveAsNewAPIHadoopFiles = NotSerializableException for Hadoop Configuration

[ https://issues.apache.org/jira/browse/SPARK-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224582#comment-14224582 ] Tathagata Das commented on SPARK-4196: -- Let me try to take a pass on this. > Stream

[jira] [Resolved] (SPARK-4344) spark.yarn.user.classpath.first is undocumented

[ https://issues.apache.org/jira/browse/SPARK-4344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-4344. -- Resolution: Fixed Fix Version/s: 1.3.0 1.2.0 > spark.yarn.user.classpa

[jira] [Commented] (SPARK-4196) Streaming + checkpointing + saveAsNewAPIHadoopFiles = NotSerializableException for Hadoop Configuration

[ https://issues.apache.org/jira/browse/SPARK-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224637#comment-14224637 ] Tathagata Das commented on SPARK-4196: -- As for now there is a workaround. You could d

[jira] [Resolved] (SPARK-4601) Call site of jobs generated by streaming incorrect in Spark UI

[ https://issues.apache.org/jira/browse/SPARK-4601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4601. -- Resolution: Fixed Fix Version/s: 1.3.0 1.2.0 > Call site of jobs gener

[jira] [Created] (SPARK-4602) saveAsNewAPIHadoopFiles by default does not use SparkContext's hadoop configuration

Tathagata Das created SPARK-4602: Summary: saveAsNewAPIHadoopFiles by default does not use SparkContext's hadoop configuration Key: SPARK-4602 URL: https://issues.apache.org/jira/browse/SPARK-4602 Pr

[jira] [Created] (SPARK-4603) EOF when broadcasting a dict with an empty string value.

Alex Angelini created SPARK-4603: Summary: EOF when broadcasting a dict with an empty string value. Key: SPARK-4603 URL: https://issues.apache.org/jira/browse/SPARK-4603 Project: Spark Issue

[jira] [Commented] (SPARK-4196) Streaming + checkpointing + saveAsNewAPIHadoopFiles = NotSerializableException for Hadoop Configuration

[ https://issues.apache.org/jira/browse/SPARK-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224649#comment-14224649 ] Sean Owen commented on SPARK-4196: -- That didn't work for me, IIRC. The problem is that th

[jira] [Commented] (SPARK-4602) saveAsNewAPIHadoopFiles by default does not use SparkContext's hadoop configuration

[ https://issues.apache.org/jira/browse/SPARK-4602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224652#comment-14224652 ] Apache Spark commented on SPARK-4602: - User 'tdas' has created a pull request for this

[jira] [Commented] (SPARK-4002) KafkaStreamSuite "Kafka input stream" case fails on OSX

[ https://issues.apache.org/jira/browse/SPARK-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224656#comment-14224656 ] Tathagata Das commented on SPARK-4002: -- [[~jerryshao] have you figured out this issue

[jira] [Commented] (SPARK-4196) Streaming + checkpointing + saveAsNewAPIHadoopFiles = NotSerializableException for Hadoop Configuration

[ https://issues.apache.org/jira/browse/SPARK-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224651#comment-14224651 ] Apache Spark commented on SPARK-4196: - User 'tdas' has created a pull request for this

[jira] [Commented] (SPARK-2072) Streaming not processing a file with particular number of entries

[ https://issues.apache.org/jira/browse/SPARK-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224660#comment-14224660 ] Tathagata Das commented on SPARK-2072: -- Does this issue still exist? If not, I am goi

[jira] [Updated] (SPARK-4598) Paginate stage page to avoid OOM with > 100,000 tasks

[ https://issues.apache.org/jira/browse/SPARK-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4598: --- Summary: Paginate stage page to avoid OOM with > 100,000 tasks (was: java.lang.OutOfMemoryErr

[jira] [Updated] (SPARK-4598) java.lang.OutOfMemoryError occurs when opening stage page of an application has 100000 tasks,

[ https://issues.apache.org/jira/browse/SPARK-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4598: --- Priority: Critical (was: Major) > java.lang.OutOfMemoryError occurs when opening stage page o

[jira] [Commented] (SPARK-907) Add JSON endpoints to SparkUI

[ https://issues.apache.org/jira/browse/SPARK-907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224811#comment-14224811 ] Andrew Ash commented on SPARK-907: -- [~dmccauley] this report looks like a duplicate of SPA

[jira] [Commented] (SPARK-4598) Paginate stage page to avoid OOM with > 100,000 tasks

[ https://issues.apache.org/jira/browse/SPARK-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224810#comment-14224810 ] Patrick Wendell commented on SPARK-4598: It is a good idea to paginate this page.

[jira] [Commented] (SPARK-2892) Socket Receiver does not stop when streaming context is stopped

[ https://issues.apache.org/jira/browse/SPARK-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224818#comment-14224818 ] Gino Bustelo commented on SPARK-2892: - Update? > Socket Receiver does not stop when s

[jira] [Commented] (SPARK-907) Add JSON endpoints to SparkUI

[ https://issues.apache.org/jira/browse/SPARK-907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224820#comment-14224820 ] David McCauley commented on SPARK-907: -- [~aash] yes, SPARK-3644 seems to be a much mor

[jira] [Closed] (SPARK-907) Add JSON endpoints to SparkUI

[ https://issues.apache.org/jira/browse/SPARK-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David McCauley closed SPARK-907. Resolution: Duplicate > Add JSON endpoints to SparkUI > - > >

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224871#comment-14224871 ] Debasish Das commented on SPARK-2426: - Actually on MovieLens dataset, I am getting goo

[jira] [Commented] (SPARK-2495) Ability to re-create ML models

[ https://issues.apache.org/jira/browse/SPARK-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224894#comment-14224894 ] Tamas Jambor commented on SPARK-2495: - hi all, what's the reason MatrixFactorizationM

[jira] [Created] (SPARK-4604) Make MatrixFactorizationModel constructor public

Xiangrui Meng created SPARK-4604: Summary: Make MatrixFactorizationModel constructor public Key: SPARK-4604 URL: https://issues.apache.org/jira/browse/SPARK-4604 Project: Spark Issue Type: Im

[jira] [Commented] (SPARK-4604) Make MatrixFactorizationModel constructor public

[ https://issues.apache.org/jira/browse/SPARK-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224973#comment-14224973 ] Apache Spark commented on SPARK-4604: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-2495) Ability to re-create ML models

[ https://issues.apache.org/jira/browse/SPARK-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224970#comment-14224970 ] Xiangrui Meng commented on SPARK-2495: -- I created SPARK-4604 for MatrixFactorizationM

[jira] [Commented] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224996#comment-14224996 ] Tianshuo Deng commented on SPARK-4452: -- [~sandyr]: Thanks for the feedback! For doub

[jira] [Created] (SPARK-4605) Spark Kernel to enable interactive Spark applications

Chip Senkbeil created SPARK-4605: Summary: Spark Kernel to enable interactive Spark applications Key: SPARK-4605 URL: https://issues.apache.org/jira/browse/SPARK-4605 Project: Spark Issue Typ

[jira] [Updated] (SPARK-4605) Spark Kernel to enable interactive Spark applications

[ https://issues.apache.org/jira/browse/SPARK-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chip Senkbeil updated SPARK-4605: - Attachment: Kernel Architecture.pdf PDF outlining the key values of the kernel and its general arc

[jira] [Updated] (SPARK-4605) Spark Kernel to enable interactive Spark applications

[ https://issues.apache.org/jira/browse/SPARK-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chip Senkbeil updated SPARK-4605: - Description: Enables applications to interact with a Spark cluster using Scala in several ways:

[jira] [Updated] (SPARK-4605) Spark Kernel to enable interactive Spark applications

[ https://issues.apache.org/jira/browse/SPARK-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chip Senkbeil updated SPARK-4605: - Attachment: Kernel Architecture Widescreen.pdf Kernel Architecture.pdf > Spark Ker

  1   2   3   >