[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239102#comment-14239102 ] Aaron Davidson commented on SPARK-4740: --- I tried to reproduce this on an EC2 cluster

[jira] [Commented] (SPARK-4793) way to find assembly jar is too strict

[ https://issues.apache.org/jira/browse/SPARK-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239076#comment-14239076 ] Apache Spark commented on SPARK-4793: - User 'adrian-wang' has created a pull request f

[jira] [Created] (SPARK-4793) way to find assembly jar is too strict

Adrian Wang created SPARK-4793: -- Summary: way to find assembly jar is too strict Key: SPARK-4793 URL: https://issues.apache.org/jira/browse/SPARK-4793 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-1350) YARN ContainerLaunchContext should use cluster's JAVA_HOME

[ https://issues.apache.org/jira/browse/SPARK-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239060#comment-14239060 ] Aniket Bhatnagar commented on SPARK-1350: - I am using hadoop 2.5.0 (CDH). Agreed t

[jira] [Resolved] (SPARK-4773) CTAS Doesn't Use the Current Schema

[ https://issues.apache.org/jira/browse/SPARK-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Ross resolved SPARK-4773. --- Resolution: Fixed Looks like this was broken by: https://github.com/apache/spark/commit/4b55482abf899

[jira] [Closed] (SPARK-4781) Column values become all NULL after doing ALTER TABLE CHANGE for renaming column names (Parquet external table in HiveContext)

[ https://issues.apache.org/jira/browse/SPARK-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianshi Huang closed SPARK-4781. > Column values become all NULL after doing ALTER TABLE CHANGE for renaming > column names (Parquet ext

[jira] [Commented] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

[ https://issues.apache.org/jira/browse/SPARK-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239006#comment-14239006 ] SUMANTH B B N commented on SPARK-3717: -- [~josephkb][~manishamde][~codedeft] I have ad

[jira] [Commented] (SPARK-3219) K-Means clusterer should support Bregman distance functions

[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238958#comment-14238958 ] Derrick Burns commented on SPARK-3219: -- Any progress on this pull request? > K-Means

[jira] [Commented] (SPARK-4785) When called with arguments referring column fields, PMOD throws NPE

[ https://issues.apache.org/jira/browse/SPARK-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238942#comment-14238942 ] Apache Spark commented on SPARK-4785: - User 'chenghao-intel' has created a pull reques

[jira] [Commented] (SPARK-4792) Add some judgments and messages on making local dir

[ https://issues.apache.org/jira/browse/SPARK-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238897#comment-14238897 ] Apache Spark commented on SPARK-4792: - User 'XuTingjun' has created a pull request for

[jira] [Created] (SPARK-4792) Add some judgments and messages on making local dir

meiyoula created SPARK-4792: --- Summary: Add some judgments and messages on making local dir Key: SPARK-4792 URL: https://issues.apache.org/jira/browse/SPARK-4792 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-4769) CTAS does not work when reading from temporary tables

[ https://issues.apache.org/jira/browse/SPARK-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4769. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3336 [https:/

[jira] [Updated] (SPARK-4769) CTAS does not work when reading from temporary tables

[ https://issues.apache.org/jira/browse/SPARK-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4769: Assignee: Cheng Hao > CTAS does not work when reading from temporary tables > --

[jira] [Created] (SPARK-4791) Create SchemaRDD from case classes with multiple constructors

Joseph K. Bradley created SPARK-4791: Summary: Create SchemaRDD from case classes with multiple constructors Key: SPARK-4791 URL: https://issues.apache.org/jira/browse/SPARK-4791 Project: Spark

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238769#comment-14238769 ] Nicholas Chammas commented on SPARK-3431: - [~nkeywal] - I took a quick look at HBa

[jira] [Resolved] (SPARK-4770) spark.scheduler.minRegisteredResourcesRatio documented default is incorrect for YARN

[ https://issues.apache.org/jira/browse/SPARK-4770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4770. --- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by pull request

[jira] [Resolved] (SPARK-3802) Scala version is wrong in dev/audit-release/blank_sbt_build/build.sbt

[ https://issues.apache.org/jira/browse/SPARK-3802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3802. --- Resolution: Fixed Assignee: Andrew Or This was fixed by [~andrewor14] in https://github.com/apa

[jira] [Updated] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3926: -- Target Version/s: 1.2.1 Fix Version/s: (was: 1.2.0) (was: 1.1.1)

[jira] [Updated] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3926: -- Affects Version/s: 1.1.1 > result of JavaRDD collectAsMap() is not serializable > --

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238721#comment-14238721 ] koert kuipers commented on SPARK-3655: -- I will update the pullrequest to put out a v

[jira] [Updated] (SPARK-4750) Dynamic allocation - we need to synchronize kills

[ https://issues.apache.org/jira/browse/SPARK-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4750: -- Labels: backport-needed (was: ) > Dynamic allocation - we need to synchronize kills > -

[jira] [Commented] (SPARK-4750) Dynamic allocation - we need to synchronize kills

[ https://issues.apache.org/jira/browse/SPARK-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238718#comment-14238718 ] Josh Rosen commented on SPARK-4750: --- Merged into {{master}} and waiting to backport into

[jira] [Updated] (SPARK-4750) Dynamic allocation - we need to synchronize kills

[ https://issues.apache.org/jira/browse/SPARK-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4750: -- Fix Version/s: 1.3.0 > Dynamic allocation - we need to synchronize kills > -

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238703#comment-14238703 ] Nicholas Chammas commented on SPARK-3431: - Here are some of the errors: {code} Ru

[jira] [Updated] (SPARK-4774) Make HiveFromSpark example more portable

[ https://issues.apache.org/jira/browse/SPARK-4774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4774: Fix Version/s: 1.2.0 > Make HiveFromSpark example more portable > --

[jira] [Resolved] (SPARK-4774) Make HiveFromSpark example more portable

[ https://issues.apache.org/jira/browse/SPARK-4774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4774. - Resolution: Fixed Assignee: Kostas Sakellis > Make HiveFromSpark example more portab

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238686#comment-14238686 ] Sean Owen commented on SPARK-3431: -- What are the errors? Problems with the tests or the t

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238685#comment-14238685 ] koert kuipers commented on SPARK-3655: -- OK that can be done. It definitely highlights

[jira] [Comment Edited] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238666#comment-14238666 ] Nicholas Chammas edited comment on SPARK-3431 at 12/8/14 11:36 PM: -

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238666#comment-14238666 ] Nicholas Chammas commented on SPARK-3431: - OK, here's a patch for {{pom.xml}} that

[jira] [Updated] (SPARK-4687) SparkContext#addFile doesn't keep file folder information

[ https://issues.apache.org/jira/browse/SPARK-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4687: - Affects Version/s: 1.2.0 > SparkContext#addFile doesn't keep file folder information > ---

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238628#comment-14238628 ] Sandy Ryza commented on SPARK-3655: --- The groupBy Iterable vs. TraversableOnce conversati

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238615#comment-14238615 ] Sean Owen commented on SPARK-3431: -- Surefire is definitely the main Maven testing plugin

[jira] [Commented] (SPARK-4417) New API: sample RDD to fixed number of items

[ https://issues.apache.org/jira/browse/SPARK-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238613#comment-14238613 ] Ilya Ganelin commented on SPARK-4417: - Hi, I'd like to work on this. Can someone pleas

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238596#comment-14238596 ] Nicholas Chammas commented on SPARK-3431: - Thanks for assigning the issue to me, J

[jira] [Updated] (SPARK-4790) Flaky test in ReceivedBlockTrackerSuite: "block addition, block to batch allocation, and cleanup with write ahead log"

[ https://issues.apache.org/jira/browse/SPARK-4790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4790: --- Assignee: Tathagata Das > Flaky test in ReceivedBlockTrackerSuite: "block addition, block to b

[jira] [Commented] (SPARK-4714) Checking block is null or not after having gotten info.lock in remove block method

[ https://issues.apache.org/jira/browse/SPARK-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238564#comment-14238564 ] Josh Rosen commented on SPARK-4714: --- For future reference, I left a big comment explaini

[jira] [Comment Edited] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238197#comment-14238197 ] Derrick Burns edited comment on SPARK-3039 at 12/8/14 10:16 PM:

[jira] [Created] (SPARK-4790) Flaky test in ReceivedBlockTrackerSuite: "block addition, block to batch allocation, and cleanup with write ahead log"

Josh Rosen created SPARK-4790: - Summary: Flaky test in ReceivedBlockTrackerSuite: "block addition, block to batch allocation, and cleanup with write ahead log" Key: SPARK-4790 URL: https://issues.apache.org/jira/brows

[jira] [Updated] (SPARK-4245) Fix containsNull of the result ArrayType of CreateArray expression.

[ https://issues.apache.org/jira/browse/SPARK-4245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-4245: --- Assignee: Takuya Ueshin > Fix containsNull of the result ArrayType of CreateArray expression. > --

[jira] [Commented] (SPARK-4737) Prevent serialization errors from ever crashing the DAG scheduler

[ https://issues.apache.org/jira/browse/SPARK-4737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238435#comment-14238435 ] Apache Spark commented on SPARK-4737: - User 'mccheah' has created a pull request for t

[jira] [Resolved] (SPARK-2175) Null values when using App trait.

[ https://issues.apache.org/jira/browse/SPARK-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2175. -- Resolution: Duplicate This is the same issue that was fixed with a warning and some better docs indeed

[jira] [Commented] (SPARK-4501) Create build/mvn to automatically download maven/zinc/scalac

[ https://issues.apache.org/jira/browse/SPARK-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238399#comment-14238399 ] Ryan Williams commented on SPARK-4501: -- I've not worked on it other than installing z

[jira] [Commented] (SPARK-4501) Create build/mvn to automatically download maven/zinc/scalac

[ https://issues.apache.org/jira/browse/SPARK-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238393#comment-14238393 ] Brennon York commented on SPARK-4501: - [~rdub] Have you started working on this? I've

[jira] [Commented] (SPARK-4789) Standardize ML Prediction APIs

[ https://issues.apache.org/jira/browse/SPARK-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238390#comment-14238390 ] Apache Spark commented on SPARK-4789: - User 'jkbradley' has created a pull request for

[jira] [Commented] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238384#comment-14238384 ] Josh Rosen commented on SPARK-3967: --- To give a quick update on this: I've merged [~preau

[jira] [Created] (SPARK-4789) Standardize ML Prediction APIs

Joseph K. Bradley created SPARK-4789: Summary: Standardize ML Prediction APIs Key: SPARK-4789 URL: https://issues.apache.org/jira/browse/SPARK-4789 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-4764) Ensure that files are fetched atomically

[ https://issues.apache.org/jira/browse/SPARK-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238368#comment-14238368 ] Josh Rosen commented on SPARK-4764: --- This has been fixed by https://github.com/apache/sp

[jira] [Updated] (SPARK-4764) Ensure that files are fetched atomically

[ https://issues.apache.org/jira/browse/SPARK-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4764: -- Fix Version/s: 1.1.2 1.3.0 > Ensure that files are fetched atomically > -

[jira] [Updated] (SPARK-4764) Ensure that files are fetched atomically

[ https://issues.apache.org/jira/browse/SPARK-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4764: -- Description: It does not seem necessary in the {{doFetchFile}} method to first download the file i

[jira] [Updated] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3431: -- Assignee: Nicholas Chammas > Parallelize execution of tests > -- > >

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238330#comment-14238330 ] Nicholas Chammas commented on SPARK-3431: - I am currently (and have been) actively

[jira] [Resolved] (SPARK-4781) Column values become all NULL after doing ALTER TABLE CHANGE for renaming column names (Parquet external table in HiveContext)

[ https://issues.apache.org/jira/browse/SPARK-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4781. - Resolution: Won't Fix This is by design, the alter table command in hive only changes meta

[jira] [Updated] (SPARK-4788) NullPointerException while launching application using spark submit in cluster deploy mode.

[ https://issues.apache.org/jira/browse/SPARK-4788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] siva venkat gogineni updated SPARK-4788: Attachment: stacktrace.txt > NullPointerException while launching application using

[jira] [Created] (SPARK-4788) NullPointerException while launching application using spark submit in cluster deploy mode.

siva venkat gogineni created SPARK-4788: --- Summary: NullPointerException while launching application using spark submit in cluster deploy mode. Key: SPARK-4788 URL: https://issues.apache.org/jira/browse/SPARK

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238289#comment-14238289 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 7:17 PM: --- I ha

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238289#comment-14238289 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 7:17 PM: --- I ha

[jira] [Updated] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4759: - Attachment: SparkBugReplicatorSmaller.scala > Deadlock in complex spark job in local mode > --

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238289#comment-14238289 ] Andrew Or edited comment on SPARK-4759 at 12/8/14 7:16 PM: --- I ha

[jira] [Updated] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4759: - Attachment: (was: SparkBugReplicatorSmaller.scala) > Deadlock in complex spark job in local mode > ---

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238289#comment-14238289 ] Andrew Or commented on SPARK-4759: -- I have a smaller reproduction for branch-1.1. It seem

[jira] [Updated] (SPARK-1600) flaky test case in streaming.CheckpointSuite

[ https://issues.apache.org/jira/browse/SPARK-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-1600: -- Affects Version/s: 1.2.0 > flaky test case in streaming.CheckpointSuite > --

[jira] [Updated] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4759: - Attachment: SparkBugReplicatorSmaller.scala > Deadlock in complex spark job in local mode > --

[jira] [Updated] (SPARK-1600) flaky test case in streaming.CheckpointSuite

[ https://issues.apache.org/jira/browse/SPARK-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-1600: -- Target Version/s: 1.3.0 (was: 1.2.0) > flaky test case in streaming.CheckpointSuite > -

[jira] [Commented] (SPARK-1600) flaky test case in streaming.CheckpointSuite

[ https://issues.apache.org/jira/browse/SPARK-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238285#comment-14238285 ] Josh Rosen commented on SPARK-1600: --- This is still flaky; here's the test result from a

[jira] [Updated] (SPARK-1600) flaky test case in streaming.CheckpointSuite

[ https://issues.apache.org/jira/browse/SPARK-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-1600: -- Affects Version/s: 1.3.0 > flaky test case in streaming.CheckpointSuite > --

[jira] [Commented] (SPARK-2175) Null values when using App trait.

[ https://issues.apache.org/jira/browse/SPARK-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238265#comment-14238265 ] Malte Buecken commented on SPARK-2175: -- It would be great to have this information so

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238221#comment-14238221 ] Davis Shepherd commented on SPARK-4759: --- The fix appears to resolve the issue in mas

[jira] [Comment Edited] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238197#comment-14238197 ] Derrick Burns edited comment on SPARK-3039 at 12/8/14 6:29 PM: -

[jira] [Comment Edited] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238197#comment-14238197 ] Derrick Burns edited comment on SPARK-3039 at 12/8/14 6:27 PM: -

[jira] [Comment Edited] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238197#comment-14238197 ] Derrick Burns edited comment on SPARK-3039 at 12/8/14 6:26 PM: -

[jira] [Comment Edited] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238197#comment-14238197 ] Derrick Burns edited comment on SPARK-3039 at 12/8/14 6:27 PM: -

[jira] [Comment Edited] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238197#comment-14238197 ] Derrick Burns edited comment on SPARK-3039 at 12/8/14 6:24 PM: -

[jira] [Commented] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238197#comment-14238197 ] Derrick Burns commented on SPARK-3039: -- Spark 1.1.1/Hadoop 1.0.4 {quote} java.lang.I

[jira] [Commented] (SPARK-4705) Driver retries in yarn-cluster mode always fail if event logging is enabled

[ https://issues.apache.org/jira/browse/SPARK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238175#comment-14238175 ] Marcelo Vanzin commented on SPARK-4705: --- It doesn't sound right to force the user to

[jira] [Created] (SPARK-4787) Resource unreleased during failure in SparkContext initialization

Jacky Li created SPARK-4787: --- Summary: Resource unreleased during failure in SparkContext initialization Key: SPARK-4787 URL: https://issues.apache.org/jira/browse/SPARK-4787 Project: Spark Issue

[jira] [Comment Edited] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238009#comment-14238009 ] Zhang, Liye edited comment on SPARK-4740 at 12/8/14 4:13 PM: -

[jira] [Updated] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang, Liye updated SPARK-4740: --- Attachment: rxin_patch-on_4_node_cluster_48CoresPerNode(Unbalance).7z Hi [~rxin], [~adav], I uploaded

[jira] [Updated] (SPARK-4001) Add Apriori algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacky Li updated SPARK-4001: Attachment: Distributed frequent item mining algorithm based on Spark.pptx [~mengxr] please check the attach

[jira] [Commented] (SPARK-1350) YARN ContainerLaunchContext should use cluster's JAVA_HOME

[ https://issues.apache.org/jira/browse/SPARK-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237914#comment-14237914 ] Thomas Graves commented on SPARK-1350: -- which version of hadoop are you using? Spar

[jira] [Created] (SPARK-4786) Parquet filter pushdown for BYTE and SHORT types

Cheng Lian created SPARK-4786: - Summary: Parquet filter pushdown for BYTE and SHORT types Key: SPARK-4786 URL: https://issues.apache.org/jira/browse/SPARK-4786 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237880#comment-14237880 ] Bertrand Bossy commented on SPARK-3039: --- @[~derrickburns]: Can you post some more in

[jira] [Commented] (SPARK-3382) GradientDescent convergence tolerance

[ https://issues.apache.org/jira/browse/SPARK-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237857#comment-14237857 ] Apache Spark commented on SPARK-3382: - User 'Lewuathe' has created a pull request for

[jira] [Commented] (SPARK-4697) System properties should override environment variables

[ https://issues.apache.org/jira/browse/SPARK-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237835#comment-14237835 ] WangTaoTheTonic commented on SPARK-4697: I took a quick look at the results filter

[jira] [Closed] (SPARK-4181) Create separate options to control the client-mode AM resource allocation request

[ https://issues.apache.org/jira/browse/SPARK-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangTaoTheTonic closed SPARK-4181. -- Resolution: Duplicate > Create separate options to control the client-mode AM resource allocatio

[jira] [Commented] (SPARK-4181) Create separate options to control the client-mode AM resource allocation request

[ https://issues.apache.org/jira/browse/SPARK-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237794#comment-14237794 ] WangTaoTheTonic commented on SPARK-4181: I see an related issue SPARK-4696 opened

[jira] [Commented] (SPARK-4705) Driver retries in yarn-cluster mode always fail if event logging is enabled

[ https://issues.apache.org/jira/browse/SPARK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237785#comment-14237785 ] WangTaoTheTonic commented on SPARK-4705: We have an configuration item "spark.even

[jira] [Commented] (SPARK-4783) System.exit() calls in SparkContext disrupt applications embedding Spark

[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237718#comment-14237718 ] David Semeria commented on SPARK-4783: -- The key idea with this proposal is that the s

[jira] [Comment Edited] (SPARK-4783) System.exit() calls in SparkContext disrupt applications embedding Spark

[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237718#comment-14237718 ] David Semeria edited comment on SPARK-4783 at 12/8/14 10:14 AM:

[jira] [Commented] (SPARK-3640) KinesisUtils should accept a credentials object instead of forcing DefaultCredentialsProvider

[ https://issues.apache.org/jira/browse/SPARK-3640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237681#comment-14237681 ] Apache Spark commented on SPARK-3640: - User 'aniketbhatnagar' has created a pull reque

[jira] [Commented] (SPARK-4785) When called with arguments referring column fields, PMOD throws NPE

[ https://issues.apache.org/jira/browse/SPARK-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237649#comment-14237649 ] Cheng Lian commented on SPARK-4785: --- Looked into this a bit. Seems that this issue is ca

[jira] [Commented] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237648#comment-14237648 ] Derrick Burns commented on SPARK-3039: -- I get the same bug when attempting to save a

[jira] [Updated] (SPARK-4785) When called with arguments referring column fields, PMOD throws NPE

[ https://issues.apache.org/jira/browse/SPARK-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4785: -- Description: Reproducible when compiled with {{-Phive-0.13.1}}, {{-Phive0.12.0}} is OK. Reproduction st

[jira] [Updated] (SPARK-4785) When called with arguments referring column fields, PMOD throws NPE

[ https://issues.apache.org/jira/browse/SPARK-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4785: -- Description: Reproducible when compiled with {{-Phive-0.13.1}}, haven't tested {{-Phive0.12.0}} yet. R

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237597#comment-14237597 ] Andrew Or commented on SPARK-4759: -- Hey I have opened the following PR to fix the symptom

[jira] [Commented] (SPARK-3154) Make FlumePollingInputDStream shutdown cleaner

[ https://issues.apache.org/jira/browse/SPARK-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237596#comment-14237596 ] Apache Spark commented on SPARK-3154: - User 'zsxwing' has created a pull request for t

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237587#comment-14237587 ] Apache Spark commented on SPARK-4759: - User 'andrewor14' has created a pull request fo