[jira] [Assigned] (SPARK-4772) Accumulators leak memory, both temporarily and permanently

[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-4772: - Assignee: Josh Rosen > Accumulators leak memory, both temporarily and permanently > -

[jira] [Updated] (SPARK-4772) Accumulators leak memory, both temporarily and permanently

[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4772: -- Assignee: Nathan Kronenfeld (was: Josh Rosen) > Accumulators leak memory, both temporarily and permanen

[jira] [Updated] (SPARK-4772) Accumulators leak memory, both temporarily and permanently

[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4772: -- Target Version/s: 1.0.3, 1.3.0, 1.1.2, 1.2.1 (was: 1.3.0, 1.2.1) > Accumulators leak memory, both tempo

[jira] [Commented] (SPARK-4812) SparkPlan.codegenEnabled may be initialized to a wrong value

[ https://issues.apache.org/jira/browse/SPARK-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240765#comment-14240765 ] Apache Spark commented on SPARK-4812: - User 'zsxwing' has created a pull request for t

[jira] [Created] (SPARK-4812) SparkPlan.codegenEnabled may be initialized to a wrong value

Shixiong Zhu created SPARK-4812: --- Summary: SparkPlan.codegenEnabled may be initialized to a wrong value Key: SPARK-4812 URL: https://issues.apache.org/jira/browse/SPARK-4812 Project: Spark Iss

[jira] [Resolved] (SPARK-1887) Update "Contributors Guide" with useful data from past threads

[ https://issues.apache.org/jira/browse/SPARK-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1887. -- Resolution: Unresolved No reply from OP. > Update "Contributors Guide" with useful data from past threa

[jira] [Resolved] (SPARK-1849) sc.textFile does not support non UTF-8 encodings

[ https://issues.apache.org/jira/browse/SPARK-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1849. -- Resolution: Won't Fix This particular issue looks like it ends in WontFix, but the idea of a "bytesFile

[jira] [Resolved] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2292. -- Resolution: Not a Problem Like SPARK-2075, it seems that this is down to different Spark code on client

[jira] [Resolved] (SPARK-2075) Anonymous classes are missing from Spark distribution

[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2075. -- Resolution: Not a Problem Fix Version/s: (was: 1.0.1) It looks like in the end this is a prob

[jira] [Resolved] (SPARK-3951) Make the external-* jars fat jars

[ https://issues.apache.org/jira/browse/SPARK-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3951. -- Resolution: Duplicate Looks like this is superseded by SPARK-3962 then, which is resolved. > Make the e

[jira] [Resolved] (SPARK-3896) checkSpeculatableTasks fask quit loop, invoking checkSpeculatableTasks is expensive

[ https://issues.apache.org/jira/browse/SPARK-3896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3896. -- Resolution: Won't Fix > checkSpeculatableTasks fask quit loop, invoking checkSpeculatableTasks is > exp

[jira] [Updated] (SPARK-3896) checkSpeculatableTasks fask quit loop, invoking checkSpeculatableTasks is expensive

[ https://issues.apache.org/jira/browse/SPARK-3896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3896: - Description: PR appears to have been withdrawn after review. Priority: Trivial (was: Major) > chec

[jira] [Commented] (SPARK-3918) Forget Unpersist in RandomForest.scala(train Method)

[ https://issues.apache.org/jira/browse/SPARK-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240732#comment-14240732 ] Sean Owen commented on SPARK-3918: -- [~josephkb] The PR you opened for this an another iss

[jira] [Resolved] (SPARK-3895) Scala style: Indentation of method

[ https://issues.apache.org/jira/browse/SPARK-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3895. -- Resolution: Won't Fix > Scala style: Indentation of method > -- > >

[jira] [Resolved] (SPARK-3869) ./bin/spark-class miss Java version with _JAVA_OPTIONS set

[ https://issues.apache.org/jira/browse/SPARK-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3869. -- Resolution: Fixed Fix Version/s: 1.2.0 The PR says it was merged, and is in the 1.2 branch. > ./

[jira] [Commented] (SPARK-4807) Add support for hadoop-2.5 + bump jets3t version

[ https://issues.apache.org/jira/browse/SPARK-4807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240715#comment-14240715 ] Sean Owen commented on SPARK-4807: -- This has been discussed a few times - the hadoop-2.4

[jira] [Resolved] (SPARK-2892) Socket Receiver does not stop when streaming context is stopped

[ https://issues.apache.org/jira/browse/SPARK-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2892. -- Resolution: Duplicate Target Version/s: (was: 1.1.0, 1.0.3) This does indeed look like the s

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240704#comment-14240704 ] Aaron Davidson commented on SPARK-4740: --- We had 4 i2.8xlarge executors each with 32

[jira] [Commented] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

[ https://issues.apache.org/jira/browse/SPARK-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240697#comment-14240697 ] SUMANTH B B N commented on SPARK-3717: -- [~josephkb][~manishamde][~codedeft] have a l

[jira] [Updated] (SPARK-4811) Custom UDTFs not working in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Santhosh updated SPARK-4811: Description: I am using the Thrift srever interface to Spark SQL and using beeline to conne

[jira] [Created] (SPARK-4811) Custom UDTFs not working in Spark SQL

Saurabh Santhosh created SPARK-4811: --- Summary: Custom UDTFs not working in Spark SQL Key: SPARK-4811 URL: https://issues.apache.org/jira/browse/SPARK-4811 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240627#comment-14240627 ] Saisai Shao commented on SPARK-4740: Hi Aaron, would you mind giving us your system co

[jira] [Updated] (SPARK-4810) Failed to run collect

[ https://issues.apache.org/jira/browse/SPARK-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] newjunwei updated SPARK-4810: - Description: my application failed like below.i want to know the possible reason.not enough memory may ca

[jira] [Comment Edited] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240579#comment-14240579 ] Aaron Davidson edited comment on SPARK-4740 at 12/10/14 3:22 AM: ---

[jira] [Updated] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-4740: --- Priority: Major (was: Blocker) > Netty's network throughput is about 1/2 of NIO's in spark-perf sortB

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240583#comment-14240583 ] Reynold Xin commented on SPARK-4740: I'm going to downgrade this ticket from blocker t

[jira] [Updated] (SPARK-4810) Failed to run collect

[ https://issues.apache.org/jira/browse/SPARK-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] newjunwei updated SPARK-4810: - Description: my application failed like below.i want to know the possible reason.not enough memory may ca

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240579#comment-14240579 ] Aaron Davidson commented on SPARK-4740: --- I spent another day trying to reproduce the

[jira] [Updated] (SPARK-4810) Failed to run collect

[ https://issues.apache.org/jira/browse/SPARK-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] newjunwei updated SPARK-4810: - Description: my application failed like below.i want to know the possible reason.not enough memory may ca

[jira] [Updated] (SPARK-4810) Failed to run collect

[ https://issues.apache.org/jira/browse/SPARK-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] newjunwei updated SPARK-4810: - Description: 2014-12-09 21:51:47,830 WARN org.apache.spark.Logging$class.logWarning(Logging.scala:71) - L

[jira] [Created] (SPARK-4810) Failed to run collect

newjunwei created SPARK-4810: Summary: Failed to run collect Key: SPARK-4810 URL: https://issues.apache.org/jira/browse/SPARK-4810 Project: Spark Issue Type: Question Reporter: newjun

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240530#comment-14240530 ] Nicholas Chammas commented on SPARK-3431: - For the record, the suite that I'm runn

[jira] [Commented] (SPARK-4809) Improve Guava shading in Spark

[ https://issues.apache.org/jira/browse/SPARK-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240525#comment-14240525 ] Apache Spark commented on SPARK-4809: - User 'vanzin' has created a pull request for th

[jira] [Commented] (SPARK-4707) Reliable Kafka Receiver can lose data if the block generator fails to store data

[ https://issues.apache.org/jira/browse/SPARK-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240521#comment-14240521 ] Apache Spark commented on SPARK-4707: - User 'harishreedharan' has created a pull reque

[jira] [Created] (SPARK-4809) Improve Guava shading in Spark

Marcelo Vanzin created SPARK-4809: - Summary: Improve Guava shading in Spark Key: SPARK-4809 URL: https://issues.apache.org/jira/browse/SPARK-4809 Project: Spark Issue Type: Improvement Af

[jira] [Comment Edited] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240519#comment-14240519 ] Nicholas Chammas edited comment on SPARK-3431 at 12/10/14 2:01 AM: -

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240519#comment-14240519 ] Nicholas Chammas commented on SPARK-3431: - OK, thanks for the updates, Sean and Ni

[jira] [Commented] (SPARK-4808) Spark fails to spill with small number of large objects

[ https://issues.apache.org/jira/browse/SPARK-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240505#comment-14240505 ] Apache Spark commented on SPARK-4808: - User 'lawlerd' has created a pull request for t

[jira] [Commented] (SPARK-4704) SparkSubmitDriverBootstrap doesn't flush output

[ https://issues.apache.org/jira/browse/SPARK-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240504#comment-14240504 ] Apache Spark commented on SPARK-4704: - User 'harishreedharan' has created a pull reque

[jira] [Created] (SPARK-4808) Spark fails to spill with small number of large objects

Dennis Lawler created SPARK-4808: Summary: Spark fails to spill with small number of large objects Key: SPARK-4808 URL: https://issues.apache.org/jira/browse/SPARK-4808 Project: Spark Issue T

[jira] [Commented] (SPARK-4707) Reliable Kafka Receiver can lose data if the block generator fails to store data

[ https://issues.apache.org/jira/browse/SPARK-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240496#comment-14240496 ] Hari Shreedharan commented on SPARK-4707: - TD and I discussed this and decided tha

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240497#comment-14240497 ] Nicolas Liochon commented on SPARK-3431: Yep, it seems ok from a maven point of vi

[jira] [Commented] (SPARK-4807) Add support for hadoop-2.5 + bump jets3t version

[ https://issues.apache.org/jira/browse/SPARK-4807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240493#comment-14240493 ] Andy Zhang commented on SPARK-4807: --- https://github.com/apache/spark/pull/3654 > Add su

[jira] [Created] (SPARK-4807) Add support for hadoop-2.5 + bump jets3t version

Andy Zhang created SPARK-4807: - Summary: Add support for hadoop-2.5 + bump jets3t version Key: SPARK-4807 URL: https://issues.apache.org/jira/browse/SPARK-4807 Project: Spark Issue Type: Improvem

[jira] [Commented] (SPARK-2892) Socket Receiver does not stop when streaming context is stopped

[ https://issues.apache.org/jira/browse/SPARK-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240437#comment-14240437 ] Ilayaperumal Gopinathan commented on SPARK-2892: It looks like this one an

[jira] [Commented] (SPARK-4806) Update Streaming Programming Guide for Spark 1.2

[ https://issues.apache.org/jira/browse/SPARK-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240416#comment-14240416 ] Apache Spark commented on SPARK-4806: - User 'tdas' has created a pull request for this

[jira] [Resolved] (SPARK-4805) BlockTransferMessage.toByteArray() trips assertion

[ https://issues.apache.org/jira/browse/SPARK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-4805. --- Resolution: Fixed Assignee: Sean Owen > BlockTransferMessage.toByteArray() trips asserti

[jira] [Updated] (SPARK-4567) Make SparkJobInfo and SparkStageInfo serializable

[ https://issues.apache.org/jira/browse/SPARK-4567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4567: -- Assignee: Sandy Ryza > Make SparkJobInfo and SparkStageInfo serializable > -

[jira] [Resolved] (SPARK-4567) Make SparkJobInfo and SparkStageInfo serializable

[ https://issues.apache.org/jira/browse/SPARK-4567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4567. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3426 [https://github.com/

[jira] [Assigned] (SPARK-3779) yarn spark.yarn.applicationMaster.waitTries config should be changed to a time period

[ https://issues.apache.org/jira/browse/SPARK-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-3779: - Assignee: Sandy Ryza > yarn spark.yarn.applicationMaster.waitTries config should be changed to a

[jira] [Updated] (SPARK-4714) BlockManager should check whether blocks have already been removed Checking block is null or not after having gotten info.lock in remove block method

[ https://issues.apache.org/jira/browse/SPARK-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4714: -- Target Version/s: 1.0.3, 1.3.0, 1.1.2, 1.2.1 (was: 1.0.3, 1.1.2, 1.2.1) > BlockManager should check whe

[jira] [Updated] (SPARK-4714) BlockManager should check whether blocks have already been removed Checking block is null or not after having gotten info.lock in remove block method

[ https://issues.apache.org/jira/browse/SPARK-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4714: -- Summary: BlockManager should check whether blocks have already been removed Checking block is null or no

[jira] [Updated] (SPARK-4714) BlockManager should check whether blocks have already been removed Checking block is null or not after having gotten info.lock in remove block method

[ https://issues.apache.org/jira/browse/SPARK-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4714: -- Fix Version/s: 1.3.0 > BlockManager should check whether blocks have already been removed Checking > bl

[jira] [Commented] (SPARK-4447) Remove layers of abstraction in YARN code no longer needed after dropping yarn-alpha

[ https://issues.apache.org/jira/browse/SPARK-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240348#comment-14240348 ] Apache Spark commented on SPARK-4447: - User 'sryza' has created a pull request for thi

[jira] [Commented] (SPARK-4714) BlockManager.dropFromMemory() should check whether block has been removed after synchronizing on BlockInfo instance.

[ https://issues.apache.org/jira/browse/SPARK-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240326#comment-14240326 ] Josh Rosen commented on SPARK-4714: --- This was fixed by https://github.com/apache/spark/p

[jira] [Updated] (SPARK-4714) BlockManager.dropFromMemory() should check whether block has been removed after synchronizing on BlockInfo instance.

[ https://issues.apache.org/jira/browse/SPARK-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4714: -- Target Version/s: 1.0.3, 1.1.2, 1.2.1 Fix Version/s: 1.1.2 1.0.3

[jira] [Updated] (SPARK-4714) BlockManager.dropFromMemory() should check whether block has been removed after synchronizing on BlockInfo instance.

[ https://issues.apache.org/jira/browse/SPARK-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4714: -- Summary: BlockManager.dropFromMemory() should check whether block has been removed after synchronizing

[jira] [Updated] (SPARK-4714) BlockManager should handle cases where block is removed after synchronizing on BlockInfo instance

[ https://issues.apache.org/jira/browse/SPARK-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4714: -- Summary: BlockManager should handle cases where block is removed after synchronizing on BlockInfo instan

[jira] [Updated] (SPARK-4714) BlockManager should handle case where block is removed after synchronizing on BlockInfo instance

[ https://issues.apache.org/jira/browse/SPARK-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4714: -- Summary: BlockManager should handle case where block is removed after synchronizing on BlockInfo instanc

[jira] [Commented] (SPARK-4159) Maven build doesn't run JUnit test suites

[ https://issues.apache.org/jira/browse/SPARK-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240283#comment-14240283 ] Apache Spark commented on SPARK-4159: - User 'srowen' has created a pull request for th

[jira] [Resolved] (SPARK-4765) Add GC back to default metrics

[ https://issues.apache.org/jira/browse/SPARK-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-4765. --- Resolution: Fixed Fix Version/s: 1.2.0 > Add GC back to default metrics > -

[jira] [Updated] (SPARK-4806) Update Streaming Programming Guide for Spark 1.2

[ https://issues.apache.org/jira/browse/SPARK-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4806: - Issue Type: Improvement (was: Bug) > Update Streaming Programming Guide for Spark 1.2 > -

[jira] [Created] (SPARK-4806) Update Streaming Programming Guide for Spark 1.2

Tathagata Das created SPARK-4806: Summary: Update Streaming Programming Guide for Spark 1.2 Key: SPARK-4806 URL: https://issues.apache.org/jira/browse/SPARK-4806 Project: Spark Issue Type: Bu

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238289#comment-14238289 ] Andrew Or edited comment on SPARK-4759 at 12/9/14 11:01 PM: I

[jira] [Updated] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4759: - Attachment: (was: SparkBugReplicatorSmaller.scala) > Deadlock in complex spark job in local mode > ---

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240238#comment-14240238 ] Andrew Or commented on SPARK-4759: -- Aha, that is a great idea. Thanks [~joshrosen] > Dea

[jira] [Commented] (SPARK-4805) BlockTransferMessage.toByteArray() trips assertion

[ https://issues.apache.org/jira/browse/SPARK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240212#comment-14240212 ] Apache Spark commented on SPARK-4805: - User 'srowen' has created a pull request for th

[jira] [Created] (SPARK-4805) BlockTransferMessage.toByteArray() trips assertion

Sean Owen created SPARK-4805: Summary: BlockTransferMessage.toByteArray() trips assertion Key: SPARK-4805 URL: https://issues.apache.org/jira/browse/SPARK-4805 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3368) Spark cannot be used with Avro and Parquet

[ https://issues.apache.org/jira/browse/SPARK-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240201#comment-14240201 ] Clay Kim commented on SPARK-3368: - I got this to work by including avro-mapred with classi

[jira] [Commented] (SPARK-4804) StringContext method to allow using Strings for column names in catalyst DSL

[ https://issues.apache.org/jira/browse/SPARK-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240194#comment-14240194 ] Apache Spark commented on SPARK-4804: - User 'culler' has created a pull request for th

[jira] [Created] (SPARK-4804) StringContext method to allow using Strings for column names in catalyst DSL

Marc Culler created SPARK-4804: -- Summary: StringContext method to allow using Strings for column names in catalyst DSL Key: SPARK-4804 URL: https://issues.apache.org/jira/browse/SPARK-4804 Project: Spark

[jira] [Issue Comment Deleted] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clay Kim updated SPARK-3039: Comment: was deleted (was: I finally managed to get this to work, using the prebuilt spark spark-1.1.1-bin-

[jira] [Commented] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240143#comment-14240143 ] Clay Kim commented on SPARK-3039: - I finally managed to get this to work, using the prebui

[jira] [Comment Edited] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240143#comment-14240143 ] Clay Kim edited comment on SPARK-3039 at 12/9/14 10:05 PM: --- I fi

[jira] [Commented] (SPARK-3477) Clean up code in Yarn Client / ClientBase

[ https://issues.apache.org/jira/browse/SPARK-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240097#comment-14240097 ] Ryan Despain commented on SPARK-3477: - As an additional note, we aren't the only peopl

[jira] [Updated] (SPARK-2450) Provide link to YARN executor logs on UI

[ https://issues.apache.org/jira/browse/SPARK-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2450: - Priority: Major (was: Minor) > Provide link to YARN executor logs on UI > ---

[jira] [Commented] (SPARK-3477) Clean up code in Yarn Client / ClientBase

[ https://issues.apache.org/jira/browse/SPARK-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240083#comment-14240083 ] Ryan Despain commented on SPARK-3477: - Question concerning the "privatization" of a fe

[jira] [Updated] (SPARK-4691) Restructure a few lines in shuffle code

[ https://issues.apache.org/jira/browse/SPARK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4691: -- Component/s: Shuffle > Restructure a few lines in shuffle code > ---

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240079#comment-14240079 ] Josh Rosen commented on SPARK-4759: --- [~andrewor14], If you wanted to run the reproducti

[jira] [Commented] (SPARK-4803) Duplicate RegisterReceiver messages sent from ReceiverSupervisor

[ https://issues.apache.org/jira/browse/SPARK-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240080#comment-14240080 ] Apache Spark commented on SPARK-4803: - User 'ilayaperumalg' has created a pull request

[jira] [Updated] (SPARK-4691) Restructure a few lines in shuffle code

[ https://issues.apache.org/jira/browse/SPARK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4691: - Target Version/s: 1.3.0, 1.2.1 Fix Version/s: 1.3.0 > Restructure a few lines in shuffle code > ---

[jira] [Created] (SPARK-4803) Duplicate RegisterReceiver messages sent from ReceiverSupervisor

Ilayaperumal Gopinathan created SPARK-4803: -- Summary: Duplicate RegisterReceiver messages sent from ReceiverSupervisor Key: SPARK-4803 URL: https://issues.apache.org/jira/browse/SPARK-4803 Pr

[jira] [Updated] (SPARK-4691) Restructure a few lines in shuffle code

[ https://issues.apache.org/jira/browse/SPARK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4691: - Labels: backport-needed (was: ) > Restructure a few lines in shuffle code > -

[jira] [Updated] (SPARK-4691) Restructure a few lines in shuffle code

[ https://issues.apache.org/jira/browse/SPARK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4691: - Summary: Restructure a few lines in shuffle code (was: code optimization for judgement) > Restructure a

[jira] [Updated] (SPARK-4691) Restructure a few lines in shuffle code

[ https://issues.apache.org/jira/browse/SPARK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4691: - Assignee: maji2014 > Restructure a few lines in shuffle code > --- > >

[jira] [Updated] (SPARK-4772) Accumulators leak memory, both temporarily and permanently

[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4772: - Affects Version/s: (was: 1.1.0) 1.0.0 > Accumulators leak memory, both temporar

[jira] [Updated] (SPARK-4772) Accumulators leak memory, both temporarily and permanently

[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4772: - Fix Version/s: (was: 1.3.0) > Accumulators leak memory, both temporarily and permanently > ---

[jira] [Updated] (SPARK-4772) Accumulators leak memory, both temporarily and permanently

[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4772: - Affects Version/s: 1.1.0 > Accumulators leak memory, both temporarily and permanently > --

[jira] [Updated] (SPARK-4772) Accumulators leak memory, both temporarily and permanently

[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4772: - Target Version/s: 1.3.0, 1.2.1 > Accumulators leak memory, both temporarily and permanently >

[jira] [Updated] (SPARK-4775) Possible problem in a simple join? Getting duplicate rows and missing rows

[ https://issues.apache.org/jira/browse/SPARK-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4775: Description: I am working on testing of HBase joins. As part of this work some simple vanil

[jira] [Updated] (SPARK-4775) Possible problem in a simple join? Getting duplicate rows and missing rows

[ https://issues.apache.org/jira/browse/SPARK-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4775: Description: I am working on testing of HBase joins. As part of this work some simple vanil

[jira] [Updated] (SPARK-4775) Possible problem in a simple join? Getting duplicate rows and missing rows

[ https://issues.apache.org/jira/browse/SPARK-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4775: Assignee: Cheng Lian > Possible problem in a simple join? Getting duplicate rows and missin

[jira] [Updated] (SPARK-4741) Do not destroy and re-create FileInputStream

[ https://issues.apache.org/jira/browse/SPARK-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4741: -- Assignee: Liang-Chi Hsieh > Do not destroy and re-create FileInputStream > -

[jira] [Updated] (SPARK-4741) Do not destroy and re-create FileInputStream

[ https://issues.apache.org/jira/browse/SPARK-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4741: -- Component/s: Spark Core Shuffle > Do not destroy and re-create FileInputStream > --

[jira] [Updated] (SPARK-4802) ReceiverInfo removal at ReceiverTracker upon deregistering receiver

[ https://issues.apache.org/jira/browse/SPARK-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilayaperumal Gopinathan updated SPARK-4802: --- Description: When the streaming receiver is deregistered, the ReceiverTracker

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239988#comment-14239988 ] Sean Owen commented on SPARK-3431: -- Hm. Well, when I run {{BlockTransferMessagesSuite}} b

[jira] [Resolved] (SPARK-874) Have a --wait flag in ./sbin/stop-all.sh that polls until Worker's are finished

[ https://issues.apache.org/jira/browse/SPARK-874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-874. --- Resolution: Fixed Fix Version/s: 1.3.0 > Have a --wait flag in ./sbin/stop-all.sh that p

[jira] [Updated] (SPARK-874) Have a --wait flag in ./sbin/stop-all.sh that polls until Worker's are finished

[ https://issues.apache.org/jira/browse/SPARK-874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-874: -- Assignee: Ben Cook (was: Archit Thakur) > Have a --wait flag in ./sbin/stop-all.sh that polls un

[jira] [Updated] (SPARK-4802) ReceiverInfo removal at ReceiverTracker upon deregistering receiver

[ https://issues.apache.org/jira/browse/SPARK-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilayaperumal Gopinathan updated SPARK-4802: --- Description: When the streaming receiver is deregistered, the ReceiverTracker

[jira] [Commented] (SPARK-4802) ReceiverInfo removal at ReceiverTracker upon deregistering receiver

[ https://issues.apache.org/jira/browse/SPARK-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239985#comment-14239985 ] Apache Spark commented on SPARK-4802: - User 'ilayaperumalg' has created a pull request

  1   2   >